Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Historical data for each IAM model region with checks of missing countries #36

Merged
merged 17 commits into from
Jan 22, 2025

Conversation

eahoegner
Copy link
Collaborator

@eahoegner eahoegner commented Jan 14, 2025

creates the mapping for aggregating the historical data from countries to to the common definitions regions for the IAMs

  • create mapping file History.yaml
  • check for missing countries (see #30)
  • do the aggregation per species and sector to IAM regions
    this may include more sub-points, but I haven't gotten around to figuring out how exactly this is done
  • resolve minor data inconsistencies
    • differences in countries available in CEDS, GFED, and the common definitions (see #30)
    • available time frames; 1997 onwards for GFED, until we use gridded data
    • unify units (GFED did not include /yr)

@eahoegner eahoegner changed the title created mapping for historical data create mapping for historical data Jan 14, 2025
@eahoegner
Copy link
Collaborator Author

so far I've only done this for CEDS alone, but GFED should work similarly. both can be added together in a next step. is there any need down the line to keep track of whether specific data came from CEDS or GFED?

@chrisroadmap
Copy link
Collaborator

so far I've only done this for CEDS alone, but GFED should work similarly. both can be added together in a next step. is there any need down the line to keep track of whether specific data came from CEDS or GFED?

My feeling is that if it's not a pain to do, it might save us a lot of headaches in the future when we come to update these datasets; so inclined to say yes

@jkikstra
Copy link
Collaborator

I've put an example of roughly what I had in mind for this in #37 (it was a lot simpler than what you started here)

@jkikstra jkikstra requested a review from znichollscr January 22, 2025 17:05
@jkikstra
Copy link
Collaborator

jkikstra commented Jan 22, 2025

Mostly everything implemented.

Ready to merge.

pyproject.toml can accept both changes.
not sure how to properly resolve pixi.lock conflict error here, and my attempt to rebase, locally, was unsuccessful (pre-commit seemed to crash and I had to do a fresh clone)

Things we can build later, on top of this:

  • create an estimate of how much of current historical emissions we'd be missing for each IAM based on the countries they miss
  • add an output metadata file describing the versions that went into the combined data set for data provenance

@znichollscr
Copy link
Collaborator

Comments and fixes in #40

@jkikstra jkikstra changed the title create mapping for historical data Historical data for each IAM model region with checks of missing countries Jan 22, 2025
@jkikstra jkikstra merged commit f752aa9 into main Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants