8. Data Source Configuration
Status
Accepted
Date
2022-11-1
Context
The current CarbonAware configuration is not intuitive for a user because it hides the relationships between entities, and so requires deep reading of the documentation to properly configure. This challenge is amplified for use-cases where different interfaces can be configured with different data sources. EG: JSON data source for emissions, but WattTime data source for forecast data.
Decision
A top-level 'DataSources' section will specify all of the data source configuration needs for the consumer.
Within that, specific data source configurations will be defined in the 'Configurations' section. Each item containing the parameters required for configuring the data source in its entirety EG: client config, file paths, additional parameters, etc.
Each data source interface can then be configured by referencing these 'Configurations' by their key.
The resulting 'DataSources' config schema being:
{
"DataSources": {
<DataSourceInterface Key1>: <'Configurations' Key String>,
<DataSourceInterface Key2>: <'Configurations' Key String>,
//...
"Configurations": {
<ConfigKey1>: {
"Type": <string enum of data source implementation>,
<Additional config key for this type>: <value>,
<Additional config key for this type>: <value>,
//...
},
<ConfigKey2>: {
//...
},
//...
}
}
}
Consequences
The configuration scheme is flexible and extensible to support any new interfaces and data sources. It reflects the relationship between data sources and all of their relevant configuration values.
Implementation
Here is an example of the proposed configuration schema change with multiple potential data source interfaces:
{
"DataSources": {
"CarbonIntensityDataSource": "WattTime",
"EnergyDataSource": "ElectricityMaps",
"EmbodiedCarbonDataSource": "ElectricityMaps",
"Configurations": {
"WattTime": {
"Type": "CarbonAware.DataSources.WattTime",
"ClientConfiguration": {
"Username": "username",
"Password": "password",
"BaseURL": "https://api2.watttime.org/v2/"
}
},
"ElectricityMaps": {
"Type": "CarbonAware.DataSources.ElectricityMaps",
"ClientConfiguration": {
"API_Key": "abcd",
"BaseURL": "https://api.electricitymap.org/v3/"
},
"disableEstimations": "true",
"emissionsFactorType": "lifecycle"
}
}
}
During initialization of the interface, the config will read to get the data
source associated with it. For example, when a CarbonIntensityDataSource
is
initialized, it will get the corresponding value from the config, which is
'WattTime' in the above example. It then looks up the value of the 'WattTime'
key in the 'Configurations' section. The object retrieved from the config will
be then used to load and configure the WattTimeDataSource
.
This "by-reference" configuration enables operators to use the same
configuration for multiple data source interfaces without requiring duplication,
as shown in the above example with the hypothetical EnergyDataSource
and
EmbodiedCarbonDataSource
both using the same ElectricityMaps
data source.
Green Impact
Neutral