API reference

The public API is exposed at the top level of the cerf package. Most users only need cerf.run(), cerf.install_package_data(), cerf.load_sample_config() and cerf.plot_siting(); the remaining modules are documented for those who want to drive individual stages of the model or extend it.

Top-level functions

The functions most users call directly. Everything else exported by cerf is documented under its module below.

cerf.run(config_file=None, config_dict=None, write_output=True, n_jobs=-1, method='sequential', initialize_site_data=None, log_level='info')

Run all CERF regions for the target year.

Parameters:

• config_file (str) – Full path with file name and extension to the input config.yml file

• config_dict (dict) – Optional instead of config_file. Configuration dictionary.

• write_output (bool) – Write output as a raster to the output directory specified in the config file

• n_jobs (int) – The number of processors to utilize. Default is -1 which uses all processors (-2 is all but one; see joblib).

• method (str) – Backend parallelization method used in Joblib. Default is sequential to manage overhead for local runs. Options for advanced configurations are: loky, threading, and multiprocessing. See https://joblib.readthedocs.io/en/latest/parallel.html for details.

• initialize_site_data –

  None if no initialization is required, otherwise either a CSV file or Pandas DataFrame of siting data bearing the following required fields:

  xcoord: the X coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  ycoord: the Y coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  retirement_year: the year (int four digit, e.g., 2050) that the power plant is to be decommissioned

  buffer_in_km: the buffer around the site to apply in kilometers

• log_level (str) – Log level. Options are ‘info’ and ‘debug’. Default ‘info’

Returns:

A data frame containing each sited power plant and their attributes

cerf.install_package_data(data_dir=None, max_attempts=5, timeout=300)

Download and unpack example data supplement from Zenodo that matches the current installed cerf distribution.

Parameters:

• data_dir (str) – Optional. Full path to the directory you wish to store the data in. Default is to install it in data directory of the package.

• max_attempts (int) – Number of download attempts before giving up (Zenodo rate limits and transient errors are retried with exponential backoff). Default 5.

• timeout (float) – Per-request timeout in seconds. Default 300.

cerf.load_sample_config(yr)

Read the config YAML file for illustrative purposes.

Parameters:

yr (int) – Target configuration year in YYYY format.

Returns:

dictionary for the configuration

cerf.config_file(yr)

Return the sample configuration file for 2010.

Parameters:

yr (int) – Target four-digit year

Returns:

Path to the target sample config file

cerf.plot_siting(result_df, boundary_shp=None, regions_shp=None, column='tech_name', markersize=5, cmap='Paired', save_figure=False, output_file=None)

Plot the results of a cerf run on a map where each technology has its own color.

Parameters:

• result_df (DataFrame) – Result data frame from running ‘cerf.run()’

• boundary_shp (str) – Full path to a boundary shapefile with file name and extension. If no file provided, the default boundary for the CONUS will be used.

• regions_shp (str) – Full path to a regions shapefile with file name and extension. If no file provided, the default regions for the CONUS will be used.

• column (str) – Column to plot

• markersize (int) – Size of power plant marker

• cmap – Custom matplotlib colormap object or name

• save_figure (bool) – If True, figure is saved to file and ‘output_file’ must be set

• output_file – If ‘save_figure’ is True, specify full path with file name and extension for the file to be saved to

class cerf.Model(config_file=None, config_dict=None, initialize_site_data=None, log_level='info', log_file=None)

Bases: ReadConfig

Model wrapper for CERF.

Parameters:

• config_file (str) – Full path with file name and extension to the input config.yml file

• config_dict (dict) – Optional instead of config_file. Configuration dictionary.

• initialize_site_data –

  None if no initialization is required, otherwise either a CSV file or Pandas DataFrame of siting data bearing the following required fields:

  xcoord: the X coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  ycoord: the Y coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  retirement_year: the year (int four digit, e.g., 2050) that the power plant is to be decommissioned

  buffer_in_km: the buffer around the site to apply in kilometers

• log_level (str) – Log level. Options are ‘info’ and ‘debug’. Default ‘info’

• log_file (str) – Optional path to a log file to write to in addition to stdout.

stage()

run model.

run_single_region(target_region_name, write_output=True)

run a single region.

Running the model

cerf.process

Entry points that stage a configuration and site every region, optionally in parallel.

Processing module for CERF

@author Chris R. vernon @email chris.vernon@pnnl.gov

License: BSD 2-Clause, see LICENSE and DISCLAIMER files

cerf.process.generate_model(config_file=None, config_dict=None, initialize_site_data=None, log_level='info')

Generate model instance for use in parallel applications.

Parameters:

• config_file (str) – Full path with file name and extension to the input config.yml file

• config_dict (dict) – Optional instead of config_file. Configuration dictionary.

• initialize_site_data –

  None if no initialization is required, otherwise either a CSV file or Pandas DataFrame of siting data bearing the following required fields:

  xcoord: the X coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  ycoord: the Y coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  retirement_year: the year (int four digit, e.g., 2050) that the power plant is to be decommissioned

  buffer_in_km: the buffer around the site to apply in kilometers

• log_level (str) – Log level. Options are ‘info’ and ‘debug’. Default ‘info’

cerf.process.region_tasks(model, data, method)

Yield the process_region keyword arguments for every region in the model.

With an in-process backend (sequential, threading) the staged full-grid arrays are shared by reference. With a process backend (loky, multiprocessing) every argument is pickled per task, so each region is cropped to its bounding box in the parent first (see RegionData.crop); a region’s payload is then proportional to its own area (Texas ~11% of the grid, Rhode Island <0.1%) instead of ~4 GB of full-grid arrays per task.

Parameters:

• model – cerf.model.Model (configuration)

• data – cerf.stage.Stage or RegionData (staged arrays)

• method – joblib backend name

cerf.process.aggregate_results(results, init_df=None)

Combine per-region results into a single sited data frame with the canonical columns and dtypes.

Parameters:

• results – Iterable of ProcessRegion / EmptyRegionResult objects (None entries are tolerated for backwards compatibility)

• init_df – Optional data frame of still-active sites from a previous run to prepend

cerf.process.cerf_parallel(model, data, write_output=True, n_jobs=-1, method='sequential')

Run all regions in parallel.

Parameters:

• model (class) – Instantiated CERF model class containing configuration options

• data (cerf.stage.Stage) – Data from cerf.stage.Stage containing NLC and suitability arrays

• write_output (bool) – Write the combined sited CSV to the output directory specified in the config

• n_jobs (int) – The number of processors to utilize. Default is -1 which uses all processors (-2 is all but one; see joblib).

• method (str) – Backend parallelization method used in Joblib. Default is sequential to manage overhead for local runs. Options for advanced configurations are: loky, threading, and multiprocessing. For the process backends each region is cropped to its bounding box before dispatch so workers receive only the data they need. See https://joblib.readthedocs.io/en/latest/parallel.html for details.

Returns:

A data frame containing each sited power plant and its attributes

cerf.process.run(config_file=None, config_dict=None, write_output=True, n_jobs=-1, method='sequential', initialize_site_data=None, log_level='info')

Run all CERF regions for the target year.

Parameters:

• config_file (str) – Full path with file name and extension to the input config.yml file

• config_dict (dict) – Optional instead of config_file. Configuration dictionary.

• write_output (bool) – Write output as a raster to the output directory specified in the config file

• n_jobs (int) – The number of processors to utilize. Default is -1 which uses all processors (-2 is all but one; see joblib).

• method (str) – Backend parallelization method used in Joblib. Default is sequential to manage overhead for local runs. Options for advanced configurations are: loky, threading, and multiprocessing. See https://joblib.readthedocs.io/en/latest/parallel.html for details.

• initialize_site_data –

  None if no initialization is required, otherwise either a CSV file or Pandas DataFrame of siting data bearing the following required fields:

  xcoord: the X coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  ycoord: the Y coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  retirement_year: the year (int four digit, e.g., 2050) that the power plant is to be decommissioned

  buffer_in_km: the buffer around the site to apply in kilometers

• log_level (str) – Log level. Options are ‘info’ and ‘debug’. Default ‘info’

Returns:

A data frame containing each sited power plant and their attributes

cerf.model

Model interface for CERF

@author Chris R. vernon @email chris.vernon@pnnl.gov

License: BSD 2-Clause, see LICENSE and DISCLAIMER files

class cerf.model.Model(config_file=None, config_dict=None, initialize_site_data=None, log_level='info', log_file=None)

Bases: ReadConfig

Model wrapper for CERF.

Parameters:

• config_file (str) – Full path with file name and extension to the input config.yml file

• config_dict (dict) – Optional instead of config_file. Configuration dictionary.

• initialize_site_data –

  None if no initialization is required, otherwise either a CSV file or Pandas DataFrame of siting data bearing the following required fields:

  xcoord: the X coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  ycoord: the Y coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003)

  retirement_year: the year (int four digit, e.g., 2050) that the power plant is to be decommissioned

  buffer_in_km: the buffer around the site to apply in kilometers

• log_level (str) – Log level. Options are ‘info’ and ‘debug’. Default ‘info’

• log_file (str) – Optional path to a log file to write to in addition to stdout.

stage()

run model.

run_single_region(target_region_name, write_output=True)

run a single region.

cerf.read_config

class cerf.read_config.ReadConfig(config_file=None, config_dict=None)

Bases: Logger

Read the configuration YAML file to a dictionary. Users can optionally pass in a configuration dictionary instead.

param config_file:

Full path with file name and extension to the input config.yml file

type config_file:

str

param config_dict:

Configuration dictionary.

type config_dict:

dict

Parameters:

config_file (str)

config_file: str

static read_yaml(yaml_file)

Read a YAML file.

get_yaml()

Read the YAML config file.

Returns:

YAML config object

get_regions_dict()

Get a dictionary of region name to region ID from the YAML file in package data.

validate_settings_files()

Ensure that files necessary files exists for settings.

validate_technology_parameters()

Validate the per-technology lifetime fields and fill defaults.

Two lifetime fields exist and are used for different purposes:

• lifetime_yrs is the economic (financing) life: the number of years over which capital costs are annuitised. It drives the annuity and levelization factors in NOV and interconnection cost.

• operational_life_yrs is the physical life: how many years a sited plant stays in service. It sets retirement_year = run_year + operational_life_yrs in the siting output and therefore controls when a plant’s footprint is released in subsequent runs that initialise from previous siting data.

The two are often equal but need not be (e.g. a 30-year financing period for a plant expected to operate 60 years). lifetime_yrs is required and must be positive. operational_life_yrs defaults to lifetime_yrs when omitted (logged at INFO) and must be positive when given.

validate_technology_files()

Ensure that files necessary files exists for technology.

validate_lmp_files()

Ensure that files necessary files exists for LMP zones.

validate_infrastructure_files()

Ensure that files necessary files exists for infrastructure.

Staging inputs

cerf.stage

Stage data for CERF run.

@author Chris R. vernon @email chris.vernon@pnnl.gov

License: BSD 2-Clause, see LICENSE and DISCLAIMER files

class cerf.stage.Stage(settings_dict, lmp_zone_dict, technology_dict, technology_order, infrastructure_dict, initialize_site_data=None)

Bases: object

Stage all spatial inputs (LMP, interconnection, NOV, NLC, suitability) for a CERF run.

Parameters:

• settings_dict (dict) – Project level settings from cerf.read_config.ReadConfig

• lmp_zone_dict (dict) – LMP zone settings from cerf.read_config.ReadConfig

• technology_dict (dict) – Technology parameters keyed by technology ID

• technology_order (list) – Technology IDs in the order used to index the 3D arrays

• infrastructure_dict (dict) – Infrastructure (substation / pipeline) settings

• initialize_site_data (str, pandas.DataFrame, None) – None, or a CSV path / DataFrame of previously sited plants

load_regions_raster()

Load the region ID raster for the CONUS into a 2D array.

load_lmp_zone_raster()

Load the lmp zoness raster for the CONUS into a 2D array.

calculate_lmp()

Calculate Locational Marginal Pricing.

calculate_ic()

Calculate interconnection costs.

calculate_nov()

Calculate Net Operational Value.

Generation and operating cost do not vary spatially; they are per-technology scalars. They are returned as read-only broadcast views with the same [tech_order, x, y] shape as nov_arr so callers can index them like any other staged array without holding two extra full-grid copies in memory.

calculate_nlc()

Calculate Net Locational Costs.

get_sited_data()

If initial condition data is provided generate an array to use unsuitable where sites and their buffers exists. Also return a data frame of active sites (not reaching retirement age) to include in the current years output.

static unsuitable_from_raster(arr, nodata=None)

Convert a suitability raster band to a boolean unsuitable mask.

The suitability convention is 0 = suitable and 1 = unsuitable. Any cell holding the raster’s declared nodata value is treated as unsuitable explicitly, so a raster whose nodata happens to be 0 cannot make missing data look suitable. Any other non-zero value is also unsuitable (logged, since it indicates a raster that is not 0/1 encoded).

Parameters:

• arr – 2D raster band

• nodata – Declared nodata value of the band, or None

Returns:

Boolean array, True where the cell is unsuitable

build_suitability_array()

Build suitability array for all technologies.

Cells are unsuitable (1) where the technology raster is non-zero or equals its declared nodata value, and, when initial siting data is provided, where an existing plant or its buffer occupies the cell.

cerf.lmp

cerf.lmp.generate_random_lmp_dataframe(n_zones=57, low_value=10, mid_value=300, high_value=500, n_samples=5000, seed=None)

Generate a random dataframe of hourly 8760 LMP values per lmp zone. Let high value LMPs only be used for 15 percent of the data. :param n_zones: Number of zones to process :param low_value: Desired minimum value of MWh :param mid_value: Desired mid value of MWh to split the 85-15 split to :param high_value: Desired max value of MWh :param n_samples: Number of intervals to split the min, max choices by :param seed: Optional seed for a local random generator; the global NumPy RNG is not used :return: Data frame of LMPs per zone

class cerf.lmp.LocationalMarginalPricing(lmp_zone_dict, technology_dict, technology_order, zones_arr)

Bases: object

Create a 3D array of locational marginal pricing per technology by capacity factor.

Locational Marginal Pricing (LMP) represents the cost of making and delivering electricity over an interconnected network of service nodes. LMPs are delivered on an hourly basis (8760 hours for the year) and help us to understand aspects of generation and congestion costs relative to the supply and demand of electricity when considering existing transmission infrastructure. LMPs are a also driven by factors such as the cost of fuel which cerf also takes into account when calculating a power plants Net Operating Value. When working with a scenario-driven grid operations model to evaluate the future evolution of the electricity system, cerf can ingest LMPs, return the sited generation per service area for the time step, and then continue this iteration through all future years to provide a harmonized view how the electricity system may respond to stressors in the future.

Parameters:

• lmp_zone_dict (dict) – A dictionary containing lmp related settings from the config file

• technology_dict (dict) – A dictionary containing technology related settings from the config file

• technology_order – A list of technologies in the order by which they should be processed

• zones_arr – An array containing the lmp zones per grid cell

static get_cf_bin(capacity_factor_fraction)

Get the correct start and through index values to average over for calculating LMP.

static zone_lookup(zones_arr, lmp_dict)

Map a zone ID array to LMP values using a vectorized lookup table.

Reproduces the semantics of np.vectorize(lmp_dict.get)(zones_arr): any zone ID present in lmp_dict receives its value and any zone ID absent from lmp_dict (including the nodata value) receives NaN. Uses a dense integer lookup table spanning the observed zone ID range, which is a single fancy-index operation instead of a Python-level call per grid cell.

Parameters:

• zones_arr (ndarray) – Integer array of LMP zone IDs per grid cell

• lmp_dict (dict) – Dictionary of {zone_id: lmp_value, …}

Returns:

Float64 array of LMP values with the same shape as zones_arr

get_lmp()

Create LMP array for the current technology.

Returns:

3D numpy array of LMP where [tech_id, x, y]

cerf.nov

class cerf.nov.NetOperationalValue(discount_rate, lifetime_yrs, unit_size_mw, capacity_factor_fraction, variable_om_esc_rate_fraction, fuel_price_esc_rate_fraction, carbon_tax_esc_rate_fraction, variable_om_usd_per_mwh, heat_rate_btu_per_kWh, fuel_price_usd_per_mmbtu, carbon_tax_usd_per_ton, carbon_capture_rate_fraction, fuel_co2_content_tons_per_btu, lmp_arr, target_year, consider_leap_year=False)

Bases: object

Calculate Net Operational Value (NOV) in ($ / yr) per grid cell for all technologies.

Parameters:

• discount_rate (float) – The time value of money in real terms. Units: fraction

• lifetime_yrs (int) – Years of the expected technology plant lifetime_yrs. Units: years

• unit_size_mw (int) – The size of the expected power plant. Units: megawatt

• capacity_factor_fraction (float) – Capacity factor defined as average annual power generated divided by the potential output if the plant operated at its rated capacity for a year. Units: fraction

• variable_om_esc_rate_fraction (float) – Escalation rate of variable cost. Units: fraction

• fuel_price_esc_rate_fraction (float) – Escalation rate of fuel. Units: fraction

• carbon_tax_esc_rate_fraction (float) – Escalation rate of carbon. Units: fraction

• variable_om_usd_per_mwh (float) – Variable operation and maintenance costs of yearly capacity use. Units: $/MWh

• heat_rate_btu_per_kWh (float) – Amount of energy used by a power plant to generate one kilowatt-hour of electricity. Units: Btu/kWh

• fuel_price_usd_per_mmbtu (float) – Cost of fuel per unit. Units: $/MMBtu

• carbon_tax_usd_per_ton (float) – The fee imposed on the burning of carbon-based fuels. Units: $/ton

• carbon_capture_rate_fraction (float) – Rate of carbon capture. Units: fraction

• fuel_co2_content_tons_per_btu (float) – CO2 content of the fuel and the heat rate of the technology. Units: tons/Btu

• lmp_arr (ndarray) – Locational Marginal Price (LMP) per grid cell for each technology in a multi-dimensional array where the shape is [tech_id, xcoord, ycoord]. Units: $/MWh

• target_year (int) – Target year of the simulation as a four digit integer (e.g., 2010)

• consider_leap_year (bool) – Choose to account for leap year in the number of hours per year calculation

Returns:

[0] generation_mwh_per_year [1] operating cost [2] NOV

HOURS_PER_YEAR_NONLEAP = 8760

HOURS_PER_YEAR_LEAP = 8784

classmethod assign_hours_per_year(target_year, consider_leap_year)

Assign the hours per year based on whether or not the target year is a leap year.

static annuity_factor_from(discount_rate, lifetime_yrs)

Calculate the annuity factor d(1 + d)^n / ((1 + d)^n - 1).

When the discount rate is zero the expression is 0/0; its limit is 1/n, which is the undiscounted case of spreading a cost evenly over the lifetime.

Parameters:

• discount_rate (float) – Real annual discount rate as a fraction

• lifetime_yrs (int, float) – Asset lifetime in years

Returns:

Annuity factor

static levelization_factor_from(escalation_rate, discount_rate, lifetime_yrs, annuity_factor)

Calculate the levelization factor for a cost stream escalating at escalation_rate.

With k = (1 + e) / (1 + d), the factor is k(1 - k^n) AF / (1 - k), the present value of the escalating stream annuitized over the lifetime. When e == d, k == 1 and the expression is 0/0; its limit is n * AF (every year’s cost has the same present value).

Parameters:

• escalation_rate (float) – Annual escalation rate of the cost as a fraction

• discount_rate (float) – Real annual discount rate as a fraction

• lifetime_yrs (int, float) – Asset lifetime in years

• annuity_factor (float) – Annuity factor for the same discount rate and lifetime

Returns:

Levelization factor

calc_annuity_factor()

Calculate annuity factor.

calc_levelization_factor_vom()

Calculate the levelizing factor for variable OM.

calc_levelization_factor_fuel()

Calculate the levelizing factor for fuel.

calc_levelization_factor_carbon()

Calculate the levelizing factor for carbon.

calc_generation()

Calculate electricity generation.

calc_nov()

Calculate NOV array for all technologies.

cerf.interconnect

class cerf.interconnect.Interconnection(template_array, technology_dict, technology_order, region_raster_file, region_abbrev_to_name_file=None, region_name_to_id_file=None, substation_file=None, transmission_costs_dict=None, transmission_costs_file=None, pipeline_costs_dict=None, pipeline_costs_file=None, pipeline_file=None, output_rasterized_file=False, output_dist_file=False, output_alloc_file=False, output_cost_file=False, interconnection_cost_file=None, output_dir=None)

Bases: object

Calculate interconnection costs per grid cell in $ / yr using:

Interconnection Cost ($ / yr) = Distance to nearest suitable transmission line (km) *

Electric grid interconnection captial cost (thous$ / km) * Annuity factor + (if gas-fired technology) Distance to nearest suitable gas pipeline (km) * Gas interconnection captial cost (thous$ / km) * Annuity factor

where, Annuity factor is (d(1 + d)**n) / ((1 + d)**n - 1) where, d = real annual discount rate (%), n = asset lifetime (years)

Parameters:

• technology_dict (dict) – Dictionary containing technology specific information from the config file

• technology_order (list) – Order of technologies to process

• region_raster_file (str) – Full path with file name and extension to the region raster file that assigns a region ID to each raster grid cell

• region_abbrev_to_name_file (str) – Deprecated and ignored; retained for backwards compatibility. Full path with file name and extension to the region abbreviation to name YAML reference file

• region_name_to_id_file (str) – Deprecated and ignored; retained for backwards compatibility. Full path with file name and extension to the region name to ID YAML reference file

• substation_file (str) – Full path with file name and extension to the input substations shapefile. If None, CERF will use the default data stored in the package.

• transmission_costs_dict (dict) – A dictionary containing the cost of connection per km to a substation having a certain minimum voltage range. Default is to load from the CERF data file ‘costs_per_kv_substation.yml’ by specifying ‘None’

• transmission_costs_file (str) – A YAML file containing the cost of connection per km to a substation having a certain minimum voltage range. Default is to load from the CERF data file ‘costs_per_kv_substation.yml’ by specifying ‘None’

• pipeline_costs_dict (dict) – A dictionary containing the cost of connection per km to a gas pipeline. Default is to load from the CERF data file ‘costs_gas_pipeline.yml’ by specifying ‘None’

• pipeline_costs_file (str) – A YAML file containing the cost of connection per km to a gas pipeline. Default is to load from the CERF data file ‘costs_gas_pipeline.yml’ by specifying ‘None’

• pipeline_file (str) – Full path with file name and extension to the input pipelines shapefile. If None, CERF will use the default data stored in the package.

• output_rasterized_file (bool) – Write distance raster; if True, set ‘output_dir’ value

• output_dist_file (bool) – Write distance raster; if True, set ‘output_dir’ value

• output_alloc_file (bool) – Write allocation file; if True, set ‘output_dir’ value

• output_cost_file (bool) – Write cost file; if True, set ‘output_dir’ value

• interconnection_cost_file (str) – Full path with file name and extension to a preprocessed interconnection cost NPY file that has been previously written. If None, IC will be calculated.

• output_dir (str) – Full path to a directory to write outputs to if desired

static calc_annuity_factor(discount_rate, lifetime_yrs)

Calculate annuity factor. Delegates to the shared implementation in cerf.nov.NetOperationalValue.annuity_factor_from() so that interconnection and NOV always use identical financial factors, including the zero-discount-rate limit.

get_pipeline_costs()

Get the costs of gas pipeline interconnection per kilometer.

process_substations()

Process input substations from shapefile.

process_pipelines()

Select natural gas pipelines data that have a length greater than 0.

Returns:

A geodataframe containing the target pipelines

static pixel_size_km(res, crs)

Return the (row, col) pixel size of a raster in kilometres.

The interconnection costs are specified in thous$/km, so the Euclidean distance to the nearest infrastructure must be measured in kilometres regardless of the raster resolution. The raster CRS must be projected (as the packaged Albers rasters are); its linear unit (metre, foot, …) is converted to kilometres. A geographic CRS has no meaningful per-pixel distance and is rejected.

Parameters:

• res (tuple) – (x_res, y_res) pixel size in CRS units, as rasterio’s dataset.res

• crs – rasterio.crs.CRS of the raster (None is rejected)

Returns:

(pixel_height_km, pixel_width_km) in array (row, col) order, suitable for scipy.ndimage.distance_transform_edt(sampling=...)

static geometries_to_shapes(geometries, values)

Yield (GeoJSON-like dict, value) pairs for rasterio.features.rasterize.

rasterize accepts shapely objects directly, but then calls __geo_interface__ on every feature, which for ~85 k features costs more than the burn itself. Points and LineStrings (the bulk of the packaged substation and pipeline data) are converted in bulk with shapely’s vectorised coordinate extraction; any other geometry type falls back to shapely.geometry.mapping.

Parameters:

• geometries – GeoPandas GeometryArray or sequence of shapely geometries

• values – Sequence of burn values, one per geometry

transmission_to_cost_raster(setting)

Create a cost per grid cell in $/km from the input GeoDataFrame of transmission infrastructure having a cost designation field as ‘_rval_’.

Distances are computed in kilometres using the raster’s pixel size, so a raster at a resolution other than 1 km produces correctly scaled costs.

Parameters:

setting (str) – Either ‘substations’ or ‘pipelines’

Returns:

Array of transmission interconnection cost per grid cell

generate_interconnection_costs_array()

Calculate the costs of interconnection for each technology.

cerf.interconnect.assign_substation_costs(gdf, transmission_costs_dict, voltage_field='min_volt')

Assign a rasterization value field (‘_rval_’) to a substation GeoDataFrame containing the cost of interconnection in thous$/km based on the voltage class bin that each substation’s minimum voltage falls in.

Parameters:

• gdf (GeoDataFrame) – Substation GeoDataFrame containing voltage_field

• transmission_costs_dict (dict) – Dictionary of {bin_id: {‘min_voltage’: int, ‘max_voltage’: int, ‘thous_dollar_per_km’: int}, …}

• voltage_field (str) – Name of the minimum voltage field. Default ‘min_volt’.

Returns:

The input GeoDataFrame with a populated ‘_rval_’ field

cerf.interconnect.preprocess_hifld_substations(substation_file, output_file=None)

Select substations from HIFLD data that are within the CONUS and either in service or under construction. A field used to rasterize (‘_rval_’) is also added containing the cost of connection in thous$/km for each substation based on its minimum voltage class.

This data is assumed to have the following fields (case-insensitive): [‘TYPE’, ‘STATE’, ‘STATUS’, ‘MIN_VOLT’]. Values in ‘TYPE’ and ‘STATUS’ are matched case-insensitively.

Parameters:

• substation_file (str) – Full path with filename and extension to the input HIFLD substation shapefile

• output_file (str) – Optional. Full path with filename and extension to the output shapefile

Returns:

A geodataframe containing the target substations

cerf.interconnect.preprocess_eia_natural_gas_pipelines(pipeline_file, output_file=None)

Select natural gas pipelines from EIA data that have a status of operating and a length greater than 0.

This data is assumed to have a ‘STATUS’ field (case-insensitive) whose values are matched case-insensitively.

Parameters:

• pipeline_file (str) – Full path with filename and extension to the input EIA pipeline shapefile

• output_file (str) – Optional. Full path with filename and extension to the output shapefile

Returns:

A geodataframe containing the target pipelines

Siting

cerf.process_region

Process a region for the target year.

@author Chris R. vernon @email chris.vernon@pnnl.gov

License: BSD 2-Clause, see LICENSE and DISCLAIMER files

class cerf.process_region.RegionData(suitability_arr, lmp_arr, generation_arr, operating_cost_arr, nov_arr, ic_arr, nlc_arr, zones_arr, xcoords, ycoords, indices_2d, regions_arr=None, region_bounds=None)

Bases: object

The staged grid arrays needed to site a region.

Bundles what cerf.stage.Stage produces so it can be handed around as one object instead of a dozen mirrored positional arguments (evaluation item 6.2). All arrays are [tech, row, col] (3D) or [row, col] (2D) over the same grid; generation_arr / operating_cost_arr may also be 1D per-technology vectors (spatially constant, see crop_to_region).

Parameters:

• suitability_arr (ndarray)

• lmp_arr (ndarray)

• generation_arr (ndarray)

• operating_cost_arr (ndarray)

• nov_arr (ndarray)

• ic_arr (ndarray)

• nlc_arr (ndarray)

• zones_arr (ndarray)

• xcoords (ndarray)

• ycoords (ndarray)

• indices_2d (ndarray)

• regions_arr (ndarray)

• region_bounds (dict)

suitability_arr: ndarray

lmp_arr: ndarray

generation_arr: ndarray

operating_cost_arr: ndarray

nov_arr: ndarray

ic_arr: ndarray

nlc_arr: ndarray

zones_arr: ndarray

xcoords: ndarray

ycoords: ndarray

indices_2d: ndarray

regions_arr: ndarray = None

region_bounds: dict = None

ARRAY_FIELDS = ('suitability_arr', 'lmp_arr', 'generation_arr', 'operating_cost_arr', 'nov_arr', 'ic_arr', 'nlc_arr', 'zones_arr', 'xcoords', 'ycoords', 'indices_2d', 'regions_arr')

classmethod field_names()

classmethod from_stage(stage)

Build from a cerf.stage.Stage (or any object exposing the same attributes).

classmethod from_kwargs(kwargs)

Pop the data fields out of a keyword-argument dict and return (RegionData, remaining_kwargs).

as_kwargs()

crop(region_id)

Return a new RegionData cropped to region_id’s bounding box (see crop_to_region).

class cerf.process_region.EmptyRegionResult(target_region_name, expansion_dict)

Bases: object

Result object for a region with no sites in its expansion plan.

Mirrors the attributes downstream code reads from a ProcessRegion result (target_region_name, run_data.sited_df, run_data.sited_dict, run_data.sited_array, run_data.expansion_dict) so callers never have to special-case None.

cerf.process_region.crop_to_region(region_id, region_bounds, suitability_arr, lmp_arr, generation_arr, operating_cost_arr, nov_arr, ic_arr, nlc_arr, zones_arr, xcoords, ycoords, indices_2d, regions_arr)

Crop every staged full-grid array to a region’s bounding box for dispatch to a worker process.

Returns a dictionary of RegionData fields holding only the region’s bounding box. The cropped arrays are contiguous copies (so pickling does not drag along the full grid), and region_bounds is rewritten so the region occupies the whole of each cropped array. Arrays that are constant over the grid (per-technology broadcast views such as generation and operating cost) are collapsed to a 1D per-technology vector.

Parameters:

• region_id (int) – Region ID as in the region raster

• region_bounds (dict) – {region_id: (ymin, ymax, xmin, xmax)} from cerf.stage.Stage

Returns:

dict of RegionData fields

class cerf.process_region.ProcessRegion(settings_dict, technology_dict, technology_order, expansion_dict, regions_dict, target_region_name, data=None, randomize=True, seed_value=0, verbose=False, write_output=False, auto_run=True, **array_kwargs)

Bases: object

Prepare a single region’s inputs and run the technology competition for it.

Construction extracts the region’s bounding box, suitability, NLC stack and metric views (all inspectable); run() performs the competition and populates run_data. Pass auto_run=False to inspect the prepared state without siting.

Parameters:

• settings_dict – Project level settings from cerf.read_config.ReadConfig

• technology_dict – Technology parameters keyed by technology ID

• technology_order – Technology IDs in array-index order

• expansion_dict – Expansion plan {region_name: {tech_id: {'n_sites': int, ...}}}

• regions_dict – {region_name: region_id}

• data – RegionData with the staged arrays (or pass the arrays as keyword arguments named as the RegionData fields; they are collected automatically)

• target_region_name – Region to process (case-insensitive)

• randomize – Random tie-breaks (True) or seeded with seed_value (False)

• seed_value – Seed used when randomize is False

• verbose – Log verbose siting information

• write_output – Write the region’s sited CSV to settings_dict['output_directory']

• auto_run – Run the competition on construction (default True)

property suitability_arr

property lmp_arr

property generation_arr

property operating_cost_arr

property nov_arr

property ic_arr

property nlc_arr

property zones_arr

property xcoords

property ycoords

property indices_2d

property regions_arr

property region_bounds

run()

Run the competition for the region and populate run_data. Idempotent.

Returns:

self

get_region_id()

Look up the region ID for the target region name.

Names are matched case-insensitively (the registry keys are lower case), so 'Rhode_Island' and 'rhode_island' resolve to the same ID.

Returns:

Corresponding region ID for the user passed region name.

get_region_bounds()

Return the grid-space bounding box (ymin, ymax, xmin, xmax) of the target region.

Uses the precomputed bounds from Stage when available; otherwise derives them from the region array.

extract_region_suitability()

Extract a single region from the suitability.

Returns a boolean array [layer, row, col] over the region’s bounding box where True marks an unsuitable cell. Layer 0 is the “no technology” default and is entirely unsuitable; layers 1..n_tech follow technology_order. Cells outside the target region are unsuitable for every technology.

mask_nlc()

Extract NLC elements for the current region with suitability applied.

Returns a plain float array where every unsuitable or NaN cell is +inf, and layer 0 (the “no technology” default) is entirely +inf so it is only chosen by argmin when no technology can site a cell.

get_grid_indices()

Generate a 1D array of grid indices for the target region to use as a way to map region level outcomes back to the full grid space.

get_grid_coordinates()

Generate 1D arrays of grid coordinates (X, Y) to use for siting based on the bounds of the target region.

extract_region_metrics()

Extract the LMP, generation, operating cost, NOV, and IC arrays for the target region and return them as dictionaries where {tech_id: 2D_region_view, …}.

The values are views into the staged full-grid arrays (no copies). Competition only reads these metrics at the handful of cells that are finally sited, so flattening five full technology stacks per region is avoided. A metric supplied as a 1D per-technology vector (spatially constant, e.g. generation) is broadcast to the region shape without allocating.

extract_lmp_zones()

Extract the lmp zones elements for the target region and return as a flat array.

competition()

Compete technologies.

cerf.process_region.process_region(target_region_name, settings_dict, technology_dict, technology_order, expansion_dict, regions_dict, data=None, randomize=True, seed_value=0, verbose=False, write_output=True, **array_kwargs)

Convenience wrapper to log time and site an expansion plan for a target region for the target year.

Parameters:

• target_region_name (str) – Name of the target region as it is represented in the region raster (matched case-insensitively).

• settings_dict (dict) – Project level setting dictionary from cerf.read_config.ReadConfig

• technology_dict (dict) – Technology level data dictionary from cerf.read_config.ReadConfig

• technology_order (list) – Technology processing order to index by from cerf.read_config.ReadConfig

• expansion_dict (dict) – Expansion plan data dictionary from cerf.read_config.ReadConfig

• regions_dict (dict) – Mapping from region name to region ID from cerf.read_config.ReadConfig

• data (RegionData) – Staged grid arrays as a RegionData (build one with RegionData.from_stage(stage)). Alternatively the individual arrays may still be passed as keyword arguments named as the RegionData fields (suitability_arr, lmp_arr, …, regions_arr, region_bounds).

• randomize (bool) – Choice to randomize when a technology has more than one NLC cheapest value

• seed_value (int) – A random seed value that is used when the user wants to replicate a run exactly

• verbose (bool) – Log verbose siting information

• write_output (bool) – Choice to write output to a file

Returns:

ProcessRegion holding the competition result in run_data (sited_df, sited_dict, sited_array, expansion_dict); an EmptyRegionResult with the same attributes and an empty sited_df if the region has no sites in its expansion plan

cerf.compete

class cerf.compete.Competition(target_region_name, settings_dict, technology_dict, technology_order, expansion_dict, lmp_dict, generation_dict, operating_cost_dict, nov_dict, ic_dict, nlc_mask, zones_arr, xcoords, ycoords, indices_flat, randomize=True, seed_value=0, verbose=False, auto_run=True)

Bases: object

Technology competition algorithm for CERF.

Grid cell level net locational cost (NLC) per technology and an electricity technology capacity expansion plan are used to compete technologies against each other to see which will win the grid cell. The technology that wins the grid cell is then sited until no further winning cells exist. Once sited, the location of the winning technology’s grid cell, along with its buffer, are no longer available for siting. The competition array is recalculated after all technologies have passed through an iteration. This process is repeated until there are either no cells left to site or there are no more power plants left to satisfy the expansion plan for any technology. For technologies that have the same NLC value in multiple grid cells that win the competition, random selection is available by default. If the user wishes to have the outcomes be repeatable, the randomizer can be set to False and a random seed set.

Parameters:

• expansion_plan (dict) – Dictionary of {tech_id: number_of_sites, …}

• nlc_mask (ndarray) – 3D array of [tech_id, x, y] for Net Locational Costs. Each technology has been masked with its suitability data, so only grid cells that are suitable have a finite NLC per tech; unsuitable cells are +inf (a numpy.ma masked array is also accepted and converted). The 0 index position is a default dimension, all +inf, which is chosen if no technologies are able to compete. A contiguous float64 input is used as the working array and is modified in place as cells are sited and excluded.

• technology_dict (dict) – A technology dictionary containing at a minimum {tech_id: buffer_in_km, …}

• randomize (bool) – Choose to make randomization of site selection where NLC is the same in multiple grid cells for a single technology random. If False, the seed_value will be used as a way to reproduce the exact siting. Default: True

• seed_value (int) – Seed for this competition’s private random number generator when randomize is False. The generator is local to the instance (the global NumPy RNG is never touched), so seeded results are identical for every joblib backend and processing order.

• verbose (bool) – Log out siting information. Default False.

run()

Run the competition once and populate sited_array / sited_df / sited_dict.

The object is prepared for inspection at construction (cheapest_arr, nlc_mask, avail_grids, …); calling run() performs the siting. It is idempotent: a second call returns the existing result.

Returns:

self

metric_at(arr, flat_index)

Return the value of a per-technology metric array at a flat region cell index.

Metric arrays may be either flat 1D arrays or 2D [row, col] views over the region’s bounding box (the latter avoids flattening full technology stacks per region); both are indexed without copying.

sited_record(tech_id, target_ix, retirement_year)

Build the output record for one sited plant as a dictionary keyed like util.empty_sited_dict().

Parameters:

• tech_id – Technology ID of the sited plant

• target_ix – Flat region cell index of the site

• retirement_year – Year the plant retires (run_year + operational_life_yrs)

add_sited_record(tech_id, target_ix, retirement_year)

Append one sited plant to self.sited_dict (a dict of column lists aligned with empty_sited_dict).

exclude_technology(tech_index)

Make every grid cell unavailable to the technology at layer tech_index.

exclude_cells(flat_indices)

Make the given flat grid cell indices unavailable to all technologies.

update_cheapest()

Recompute the cheapest technology per grid cell (0 where no technology is available).

log_outcome()

Log a warning sites that were not able to be sited.

compete()

Outputs and utilities

cerf.outputs

cerf.outputs.plot_siting(result_df, boundary_shp=None, regions_shp=None, column='tech_name', markersize=5, cmap='Paired', save_figure=False, output_file=None)

Plot the results of a cerf run on a map where each technology has its own color.

Parameters:

• result_df (DataFrame) – Result data frame from running ‘cerf.run()’

• boundary_shp (str) – Full path to a boundary shapefile with file name and extension. If no file provided, the default boundary for the CONUS will be used.

• regions_shp (str) – Full path to a regions shapefile with file name and extension. If no file provided, the default regions for the CONUS will be used.

• column (str) – Column to plot

• markersize (int) – Size of power plant marker

• cmap – Custom matplotlib colormap object or name

• save_figure (bool) – If True, figure is saved to file and ‘output_file’ must be set

• output_file – If ‘save_figure’ is True, specify full path with file name and extension for the file to be saved to

cerf.utils

cerf.utils.region_bounding_boxes(regions_arr)

Compute the grid-space bounding box of every region ID present in a 2D region raster array.

The bounds follow Python slice conventions so that arr[ymin:ymax, xmin:xmax] is the smallest window containing every cell of the region, matching the values previously derived per region with np.where.

Parameters:

regions_arr (ndarray) – 2D array of integer region IDs

Returns:

Dictionary of {region_id: (ymin, ymax, xmin, xmax)} for every region ID that occurs in the array (a nodata/background ID is included if present)

cerf.utils.results_to_geodataframe(result_df, target_crs)

Convert the results from ‘cerf.run()’ to a GeoDataFrame.

Parameters:

• result_df (DataFrame) – Result data frame from running ‘cerf.run()’

• target_crs – Coordinate reference system to assign the output.

Returns:

GeoPandas GeoDataFrame of results

cerf.utils.kilometers_to_miles(input_km_value)

Convert kilometers to miles.

Parameters:

input_km_value (float, int) – Kilometer value to convert to miles

Returns:

Miles

cerf.utils.empty_sited_dict()

Initialize a sited dictionary.

cerf.utils.sited_dtypes()

Return the data type of every column produced by empty_sited_dict().

Keeping this complete ensures an empty initial frame, per-region results, and a CSV round trip through ingest_sited_data all carry identical dtypes rather than whatever pandas infers per column.

cerf.utils.default_suitability_files()

Return a dictionary of default suitability file names keyed by technology name.

cerf.utils.default_suitabiity_files()

Deprecated misspelling of default_suitability_files(); kept for backwards compatibility.

cerf.utils.buffer_window(target_index, nrows, ncols, ncells)

Return the (row_slice, col_slice) of the square window of ncells cells around a flat grid index, clipped to the grid.

Parameters:

• target_index (int) – Flat (row-major) index of the target cell

• nrows (int) – The number of rows in the parent 2D array

• ncols (int) – The number of columns in the parent 2D array

• ncells (int) – The number of cells for the buffer extending as a radius

Returns:

(slice(r0, r1), slice(c0, c1)) usable directly on the 2D array

cerf.utils.buffer_flat_indices(target_index, nrows, ncols, ncells)

Return the sorted flat indices of the square window of ncells cells around a flat grid index, clipped to the grid, as an integer NumPy array.

Parameters are as for buffer_window.

cerf.utils.buffer_flat_array(target_index, arr, nrows, ncols, ncells, set_value)

Assign a value to the neighboring elements of a 1D array as if they were in 2D space. The number of neighbors are based on the ncells argument which is used to define the window around the target cell to be altered as if they were in 2D space.

Parameters:

• target_index (int) – Index of the target element in the 1D array

• arr (ndarray) – A 1D array that has been flattened from a corresponding 2D array; modified in place

• nrows (int) – The number of rows in the parent 2D array

• ncols (int) – The number of columns in the parent 2D array

• ncells (int) – The number of cells for the buffer extending as a radius

• set_value (int; float) – The value to set for the selected buffer

Returns:

[0] Modified 1D array (the same object as arr) [1] Sorted integer array of buffered flat indices

cerf.utils.array_to_raster(arr, template_raster_file, output_raster_file)

Write a raster file from a 2D array.

cerf.utils.raster_to_coord_arrays(template_raster)

Use the template raster to create two 2D arrays containing the X and Y cell-centre coordinates of every grid cell, computed from the raster’s affine transform.

Parameters:

template_raster (str) – Full path with file name and extension to the input raster.

Returns:

[0] 2D array of X coordinates [1] 2D array of Y coordinates

cerf.utils.ingest_sited_data(run_year, x_array, siting_data, template_raster_file)

Import sited data containing the locations and additional data to establish an initial suitability condition representing power plants and their siting buffer.

Required fields are the following and they can appear anywhere in the CSV or data frame:

xcoord: the X coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003) ycoord: the Y coordinate of the site in meters in USA_Contiguous_Albers_Equal_Area_Conic (EPSG: 102003) retirement_year: the year (int four digit, e.g., 2050) that the power plant is to be decommissioned buffer_in_km: the buffer around the site to apply in kilometers

Parameters:

• run_year (int) – Four-digit year of the current run (e.g., 2050)

• x_array (ndarray) – 2D array of X coordinates for the entire grid space

• siting_data (str, DataFrame) – Full path with file name and extension for the input siting file or a Pandas DataFrame

• template_raster_file (str) – Full path with file name and extension to the input template raster file containing a grid index value per grid cell.

Returns:

[0] 2D array of 0 (suitable) and 1 (unsuitable) values where 1 are the sites and their buffers of active power plants

[1] Pandas DataFrame of active sites (not retired)

cerf.package_data

cerf.package_data.config_file(yr)

Return the sample configuration file for 2010.

Parameters:

yr (int) – Target four-digit year

Returns:

Path to the target sample config file

cerf.package_data.cerf_regions_raster()

Return the cerf regions raster file.

cerf.package_data.cerf_regions_shapefile()

Return the cerf regions shapefile as a Geopandas data frame. Used in output plot.

cerf.package_data.cerf_boundary_shapefile()

Return the cerf boundary shapefile as a Geopandas data frame. Used in output plot.

cerf.package_data.cerf_crs()

Return a coordinate reference system (CRS) object of class ‘pyproj.crs.crs.CRS’ for USA_Contiguous_Albers_Equal_Area_Conic.

cerf.package_data.get_default_gas_pipelines()

Return the full path with file name and extension to the default gas pipeline shapefile

cerf.package_data.get_costs_per_kv_substation_file()

Return the full path with file name and extension to the default costs per km of each kv substation file.

cerf.package_data.get_costs_gas_pipeline()

Return the full path with file name and extension to the default costs per km to gas connect to pipelines.

cerf.package_data.costs_per_kv_substation()

Return a dictionary of the cost of interconnection to substations of certain KV classes.

cerf.package_data.load_sample_config(yr)

Read the config YAML file for illustrative purposes.

Parameters:

yr (int) – Target configuration year in YYYY format.

Returns:

dictionary for the configuration

cerf.package_data.list_available_suitability_files()

Return a list of available suitability files.

cerf.package_data.sample_lmp_zones_raster_file()

Return path for the sample lmp zoness raster file.

cerf.package_data.get_sample_lmp_file()

Return the sample 8760 hourly locational marginal price sample file.

cerf.package_data.get_sample_lmp_data()

Return the sample 8760 hourly locational marginal price data as a Pandas DataFrame.

cerf.package_data.get_suitability_raster(default_raster)

Return the default suitability raster file associated with the technology being processed.

cerf.package_data.get_region_abbrev_to_name_file()

Return the file path for region abbreviation to region name.

cerf.package_data.get_region_abbrev_to_name()

Return a dictionary of region abbreviation to region name.

cerf.package_data.get_region_name_to_id()

Return the region name to ID file path.

cerf.package_data.get_data_directory()

Return the directory of where the cerf package data resides.

cerf.package_data.get_substation_file()

Return the default substation file for the CONUS.

cerf.install_supplement

class cerf.install_supplement.InstallSupplement(data_dir=None, max_attempts=5, timeout=300, backoff_seconds=5)

Bases: object

Download and unpack example data supplement from Zenodo that matches the current installed cerf distribution.

Parameters:

• data_dir (str) – Optional. Full path to the directory you wish to store the data in. Default is to install it in data directory of the package.

• max_attempts (int) – Number of download attempts before giving up. Zenodo enforces rate limits (HTTP 429) and occasionally returns transient 5xx errors, both of which are retried with exponential backoff. Default 5.

• timeout (float) – Per-request timeout in seconds. Default 300.

• backoff_seconds (float) – Initial backoff between attempts in seconds; doubles each retry. A Retry-After header from the server takes precedence when present. Default 5.

DATA_VERSION_URLS = {'2.0.0': 'https://zenodo.org/records/5218436/files/cerf_package_data.zip?download=1', '2.0.1': 'https://zenodo.org/records/5218436/files/cerf_package_data.zip?download=1', '2.0.2': 'https://zenodo.org/records/5218436/files/cerf_package_data.zip?download=1', '2.0.3': 'https://zenodo.org/records/5218436/files/cerf_package_data.zip?download=1', '2.0.4': 'https://zenodo.org/records/5247690/files/cerf_package_data.zip?download=1', '2.0.5': 'https://zenodo.org/records/5247690/files/cerf_package_data.zip?download=1', '2.0.6': 'https://zenodo.org/records/5247690/files/cerf_package_data.zip?download=1', '2.0.7': 'https://zenodo.org/records/5514010/files/cerf_package_data.zip?download=1', '2.0.8': 'https://zenodo.org/records/5514010/files/cerf_package_data.zip?download=1', '2.0.9': 'https://zenodo.org/records/5514010/files/cerf_package_data.zip?download=1', '2.1.0': 'https://zenodo.org/records/5514010/files/cerf_package_data.zip?download=1', '2.1.1': 'https://zenodo.org/records/5514010/files/cerf_package_data.zip?download=1', '2.2.0': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.2.1': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.3': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.3.1': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.3.2': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.3.3': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.4.0': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.4.1': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1', '2.5.0': 'https://zenodo.org/records/6998151/files/cerf_package_data.zip?download=1'}

RETRYABLE_STATUS_CODES = (429, 500, 502, 503, 504)

classmethod get_data_link(current_version)

Return the data URL for a cerf version.

An exact entry in DATA_VERSION_URLS wins. Otherwise the URL registered for the newest version not newer than the installed one is used, with a warning, so a patch or minor release that did not change the data supplement keeps working without a code change here. Versions older than every registered entry, or unparsable versions, raise KeyError as before.

Parameters:

current_version (str) – Installed cerf version string

Returns:

URL to the versioned data archive

download(data_link)

Download the archive at data_link and return its bytes.

Retries on connection errors, timeouts, and retryable HTTP status codes with exponential backoff. Raises RuntimeError with the URL, final status, content type, and a snippet of the body if the server never returns a ZIP archive. This replaces the previous behaviour of handing whatever bytes came back straight to zipfile, which surfaced rate-limit HTML pages as an opaque BadZipFile.

Parameters:

data_link (str) – URL to download

Returns:

Archive content as bytes

static extract(content, data_directory)

Extract every file in the archive into data_directory, flattening any parent directories.

Parameters:

• content (bytes) – ZIP archive content

• data_directory (str) – Destination directory

Returns:

List of extracted file paths

fetch_zenodo()

Download and unpack the Zenodo example data supplement for the current cerf distribution.

cerf.install_supplement.install_package_data(data_dir=None, max_attempts=5, timeout=300)

Download and unpack example data supplement from Zenodo that matches the current installed cerf distribution.

Parameters:

• data_dir (str) – Optional. Full path to the directory you wish to store the data in. Default is to install it in data directory of the package.

• max_attempts (int) – Number of download attempts before giving up (Zenodo rate limits and transient errors are retried with exponential backoff). Default 5.

• timeout (float) – Per-request timeout in seconds. Default 300.

cerf.logger

Logger for CERF model.

Copyright (c) 2018, Battelle Memorial Institute

Open source under license BSD 2-Clause - see LICENSE and DISCLAIMER

@author: Chris R. Vernon (chris.vernon@pnnl.gov)

class cerf.logger.Logger

Bases: object

Manage the package-wide cerf logger.

All CERF modules log through logging.getLogger(__name__), which makes them children of the cerf logger. Handlers are attached to that named logger only, so CERF never modifies the root logger, never changes the level of third-party libraries and never removes handlers it did not create. Attaching handlers is idempotent: creating several models in one session does not duplicate log lines.

LOGGER_NAME = 'cerf'

CONSOLE_HANDLER_NAME = 'cerf-console'

FILE_HANDLER_NAME = 'cerf-file'

LOG_FORMAT_STRING = '%(asctime)s - %(name)s - %(levelname)s - %(message)s'

property log_format

Generate log formatter.

property logger

The cerf package logger.

static resolve_level(log_level)

Translate the user-facing log_level string (‘info’ or ‘debug’) to a logging level.

initialize_logger(log_level='info', log_file=None)

Attach the console handler and, optionally, a file handler.

Parameters:

• log_level (str) – Log level. Options are ‘info’ and ‘debug’. Default ‘info’

• log_file (str) – Optional path to a log file. If given, log records are also written there.

console_handler(log_level='info')

Attach a stdout handler to the cerf logger (idempotent).

file_handler(log_level='info', log_file=None)

Attach a file handler to the cerf logger (idempotent).

classmethod close_logger()

Detach and close the handlers CERF attached to the cerf logger.

Handlers owned by the application (on the root logger or elsewhere) are left untouched.