Add CEDS processing

[znichollscr]

@jkikstra nice and simple one I think. Seems to run smoothly with no issue.

@chrisroadmap @jkikstra a few questions:

1. For the regional column, do we want to use ISO codes or country names (CEDS uses ISO codes by default I think, which seems sensible to me, but now is a good time to check)?
2. Do we want to split saving so that global totals are separate from country totals? Or do we not care? (in the current data folder setup, they are separate so maybe that question is already answered? Ignore me that split is to do with raw data, not processed data)
3. Do we want to put bunkers in global totals or countries or report separately or something else?

[jkikstra]

Coming back to this tomorrow, but quick answers:

1. i like iso3c. Note that in common-definitions, they use names by default, but I think iso3c is easier because it is more standardized than names (think cases like "bolivia, plurinational state of"). And we have the start of the solution there by creating ISO codes there too; Create aggregate regions for each IAM #25 (comment).
2. No strong feelings, as long as both are there, and in exactly the same format such that binding them is easy.
3. International shipping (and Aircraft too) is harmonized only at the global level; so in global totals is fine with me.

[chrisroadmap]

1. Good to converge on the ISO3 v full names issue here.
2. I put the World totals at the top of the national processed data if for no other reason than it was what I cared about most and I was too lazy to write out another file. But as with Jarmo, no strong opinions.

[znichollscr]

Ok great, this should be ready for you then @jkikstra

[jkikstra]

Great work @chrisroadmap and @znichollscr.

Things I note:

1. Double-checked the sector mapping in "CEDS Mapping 2024".
   It resparked some questions, which I put in CEDS sector mapping: Solvents for non-VOC #28 (no change for CEDS, maybe for preprocessing IAM data before harmonization later) and CEDS Sector Mapping: 7BC_Indirect-N2O-non-agricultural #29.
   For CEDS Sector Mapping: 7BC_Indirect-N2O-non-agricultural #29, suggesting to put "7BC_Indirect-N2O-non-agricultural" under "Agriculture", and thus also in the full total. @chrisroadmap and @znichollscr what do you think for global totals; should '7BC_Indirect-N2O-non-agricultural' be included?
   (meaning we only exclude '6B_Other-not-in-total').

2. Checked country_combinations in SSP data.

• Serbia/kosovo
  • pop: no kosovo
  • gdp: no kosovo
• Israel/Palestine
  • pop: both
  • gdp: no palestine. I have been in discussions with Rob Dellink; palestine projections will be added.
• Sudan/South Sudan
  • pop: both
  • gdp: both

So, I suggest that once we move to the updated/final new SSP projections in the harmonization, we do:

• remove isr_pse aggregation from CEDS.
• remove sdn_ssd aggregation from CEDS.
• keep srb_ksv aggregation from CEDS.
  NOTE 1: a more comprehensive check on the coverage of countries&territories of each datasets (CEDS, GFED, SSP_GDP, SSP_POP, each IAM region-to-country mapping) still needs to be done.
  NOTE 2: if need be, we could choose to not use the CEDS country data directly but create our own country list from a different country-mask based on the gridded files (as for GFED). We may want to do this in any case, to double-check that both methods give the same data.
  I've put this under a new issue Country combinations: which countries are available, in which datasets, and where are combinations necessary? #30

3. Units
   We now do unit_wishes all Mt/yr.
   I note that common-definitions has kt N2O/yr for NOx.
   Could you add this conversion here?

4. Checking aggregation of countries to global
   Can we add a check to see where/whether for each sector sum(national)==global ; and that this is as expected?

5. other, tiny things
   a. ceds.attrs["name"] = "CEDS21" - delete? Or is this used anywhere?

[jkikstra]

Did 1 and 5 (+3 partially; unit wish, but not the unit conversion) in #31; please feel free to merge that in and continue.

[znichollscr]

1. @chrisroadmap and @znichollscr what do you think for global totals; should '7BC_Indirect-N2O-non-agricultural' be included?

My instinct is yes

1. meaning we only exclude '6B_Other-not-in-total'

Sorry if this is answered already, I would have assumed we include this too...

• I've put this under a new issue Country combinations: which countries are available, in which datasets, and where are combinations necessary? #30

Sounds good. My instinct would be to do the masking ourselves based on CEDS gridded data (that gives us the most control and means we can use the same mask for CEDS and BB4CMIP).

Did 1 and 5 (+3 partially; unit wish, but not the unit conversion) in #31; please feel free to merge that in and continue

Perfect thanks.

[znichollscr]

3. Units

Hacked in now

[znichollscr]

4. Can we add a check to see where/whether for each sector sum(national)==global ; and that this is as expected?

Done (surprisingly clean, different to last time :) )

[znichollscr]

Alrighty @jkikstra I would suggest merging this. We can update when resolving #28 #29 #30 #32 separately. Hit the merge button if you agree

[jkikstra]

OK, I found two things, which I will now work on committing, before merging.

1. I made a mistake above where i mentioned unit-wishes; it should of course have been "kt N2O" for "N2O" (not for NOx!). So this conversion actually is easier (just factor 1000). Sorry for making you do unnecessary work!
2. ceds has a region "global", which should only have aircraft and international shipping; but somehow now has small amounts of "Industrial Sector" for SO2, NO2, and CO. It does not have aviation by country. I'll make this more clear in our notebook.

[znicholls]

1. Sorry for making you do unnecessary work!

All good, just rip out those weird paths then and should run smoothly...

2. I'll make this more clear in our notebook

Nice

[chrisroadmap]

1. I made a mistake above where i mentioned unit-wishes; it should of course have been "kt N2O" for "N2O"

Now, I wonder if I'm the only person that would prefer to see N2O reported in Mt rather than kt. The numbers are more tractable (order 10 Mt/yr not 10000 kt/yr), it's consistent with CH4 and the SLCFs, and it has a natural synergy with the ppb atmospheric concentration unit. Can we do this?

[znichollscr]

Now, I wonder if I'm the only person that would prefer to see N2O reported in Mt rather than kt. The numbers are more tractable (order 10 Mt/yr not 10000 kt/yr), it's consistent with CH4 and the SLCFs, and it has a natural synergy with the ppb atmospheric concentration unit. Can we do this?

I think we should do this for RCMIP. For this, which is targeted at IAM people, I would just stick with what IAM teams are used to in order to minimise headaches.

[jkikstra]

Agree with Zeb.
While I also think it is nice to just have all in Mt, I don't like the idea of asking all teams to update their numbers from kt to Mt (perfectly doable, of course, but easier just for us to do it), and this here is the place to do it I think.