Audit of model outputs

[tsmbland]

WORK IN PROGRESS

Doing an audit of the outputs defined in outputs/mca.py to figure out what everything is doing and decide what we need to keep and what we can get rid of.

NOTES:

My initial feeling is that we can probably get rid of metric_supply, metricy_supply, metric_consumption, metricy_consumption, AggregateResources and FiniteResources.

The costs outputs can also be tidied and homogenised. Some functions are recalculating supply and consumption, some are taking them directly from market - presumably all of them could be doing the latter. Also a lot of the maths is repeated multiple times throughout the codebase. For example we have three functions for calculating LCOE. Why not one function that is called in three places?

The code in this module is the opposite of DRY (wet?). For example, capacity_to_service_demand() is defined (identically) in both sector_lcoe and sector_eac. This really needs to be improved

The metricy_supply, metricy_consumption and all the costs outputs fail with the same error: ValueError: cannot insert hour, already exists

The format of the costs files is very heterogeneous. we should standardise this - what columns do we need in these files and what can we leave out?

What is "dst_region"?

Lots of files have a "units_prices" column which isn't necessary

---

SUPPLY

supply

• takes data directly from market
• gives the total supply of each commodity in each region, timeslice and year
• should (need to check) match with data in sector folders. However, the data in sector folders is disaggregated over technology, asset and (obviously) sector
• columns: timeslice, commodity, region, year, units_prices, month, day, hour, supply

metric_supply (registered as "timeslice_supply")

• calls the sector_supply function
• this recalculates supply using the muse.production.supply function, which calls the muse.quantities.supply function - this is the function used for the adhoc solver which doesn't include the minimum service constraint
• columns: agent, asset, category, comm_usage, commodity, day, dst_region, hour, installed, month, region, sector, supply, technology, timeslice, units_prices, year
• (files in sector folders have: timeslice, commodity, year, asset, region, month, day, hour, technology, installed, supply)
• The only thing I think could be useful compared to the sector files is the agent column - so let's find a way to put this in the sector files, then we can delete this output

metricy_supply (registered as "yearly_supply")

• calls the sectory_supply function
• aggregates capacity data, and then recalculates supply using the same function as above
• I guess this also combines data for all timeslices to provide a yearly aggregate (need to check)
• I get an error when I try to use it
• With old MUSE: output is identical to metric_supply -> can definitely delete this

---

CONSUMPTION

consumption

• takes data directly from market
• gives the total consumption of each commodity in each region, timeslice and year
• columns: timeslice, commodity, region, year, units_prices, month, day, hour, consumption

metric_consumption (registered as "timeslice_consumption")

• calls the sector_consumption function
• recalculates both supply (using muse.production.supply, as above) and consumption (using muse.quantities.consumption)
• columns: agent, asset, category, comm_usage, commodity, consumption, day, dst_region, hour, installed, month, region, sector, technology, timeslice, units_prices, year
• similar deal to above re agent column

metricy_consumption (registered as "yearly_consumption")

• similar to metricy_supply

---

PRICES

prices

• takes data directly from market
• gives the price of each commodity in each timeslice, region and year
• columns: timeslice, commodity, region, year, units_prices, month, day, hour, prices

---

CAPACITY

capacity

• takes data directly from sectors list
• gives the capacity of each asset (although no asset column)
• a few operations in the sector_capacity function. Look like this isn't recalculating anything but just summing data across assets, although I'm confused as I recall the capacity files having multiple rows for different assets. Look into this.
• ideally we'd have an asset column
• columns: agent, capacity, dst_region, installed, region, sector, technology, type, year

---

COSTS

metric_fuel_costs (registered as "fuel_costs")

• calls the sector_fuel_costs function
• recalculates supply and consumption, and then recalculates fuel costs based on consumption
• columns (old MUSE): agent, asset, category, comm_usage, commodity, fuel_consumption_costs, installed, region, sector, technology, timeslice, unit_prices, year

metric_capital_costs (registered as "capital_costs")

• calls the sector_capital_costs function
• gets capacity data directly from sectors, and then recalculates capital costs based on capacity. Also gets consumption data directly from market, but this isn't used in the calculations.
• columns (old MUSE): agent, asset, capital_costs, category, index, installed, region, sector, technology, timeslice, year

metric_emission_costs (registered as "emission_costs")

• calls the sector_emission_costs function
• recalculates supply, and then recalculates fuel costs based on supply
• Old MUSE: doesn't work

metric_lcoe (registered as "LCOE")

• calls the sector_lcoe function
• recalculates LCOE from scratch. Needs a closer look
• columns (old MUSE): LCOE, agent, category, region, sector, technology, timeslice, year

metric_eac (registered as "EAC")

• calls the sector_eac function
• recalculates EAC from scratch. Needs a closer look
• columns (old MUSE): EAC, agent, category, region, sector, technology, timeslice, year

---

OTHER

AggregateResources

• not sure what this does. Doesn't seem to be used, and I get an error when I try to use it.
• Doesn't even work with old MUSE
• I imagine we can delete

FiniteResources (registered as "finite_resources")

• as above

[tsmbland]

@dalonsoa Some very rough notes about the output files. Happy deleting!