Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GFED4 BB4CMIP download and basic handling #13

Merged
merged 3 commits into from
Oct 22, 2024
Merged

Add GFED4 BB4CMIP download and basic handling #13

merged 3 commits into from
Oct 22, 2024

Conversation

znichollscr
Copy link
Collaborator

No description provided.

@znichollscr znichollscr requested a review from jkikstra October 17, 2024 06:42
@jkikstra jkikstra mentioned this pull request Oct 22, 2024
Copy link
Collaborator

@jkikstra jkikstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the esgpull did not run successfully for me.

(esgf) C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical>esgpull download
<8b2ffd>
└── tags:        bb4cmip
    distrib:     False
    latest:      True
    replica:     None
    retracted:   False
    grid_label:  gn
    mip_era:     CMIP6Plus
    project:     input4MIPs
    source_id:   DRES-CMIP-BB4CMIP7-1-0
    variable_id: BC, CH4, CO, CO2, N2O, NH3, NOx, OC, SO2, gridcellarea
    files:       0 bytes / 11.1 GiB [0/19]
<2c3e51> · 537.3 MiB · 942.3 kiB/s · esgf-data2.llnl.gov
<30fd96> · 537.7 MiB · 762.5 kiB/s · esgf-data2.llnl.gov
<50cca8> · 719.8 MiB · 849.6 kiB/s · esgf-data2.llnl.gov
<013eba> · 719.8 MiB · 802.3 kiB/s · esgf-data2.llnl.gov
<51af2e> · 719.7 MiB · 772.9 kiB/s · esgf-data2.llnl.gov
<61c679> · 538.2 MiB · 1,021.2 kiB/s · esgf-data2.llnl.gov
<cfa534> · 41.2 kiB · 203.1 kiB/s · esgf-data2.llnl.gov
<62f9ea> · 537.5 MiB · 875.2 kiB/s · esgf-data2.llnl.gov
<75178b> · 537.8 MiB · 877.4 kiB/s · esgf-data2.llnl.gov
<86e383> · 538.5 MiB · 920.0 kiB/s · esgf-data2.llnl.gov
<dc1fbb> · 537.7 MiB · 1.1 MiB/s · esgf-data2.llnl.gov
<cd953c> · 719.4 MiB · 1.0 MiB/s · esgf-data2.llnl.gov
<f3d829> · 719.9 MiB · 1.7 MiB/s · esgf-data2.llnl.gov
  13/13 29:46
Downloaded 13 new files for a total size of 7.2 GiB
ExceptionGroup: Download (6 sub-exceptions)
See C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical\data\national\gfed-bb4cmip\data_raw\log\esgpull-download-2024-10-22_11-32-36.log for error log.
Aborted!

Any tips/instructions on how to still complete the download? Or how to restart?

# unit_label = "Mt CH4 / yr"
# rcmip_gas = "CH4"

# # TODO: check. I don't think we used this last time.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably because it was only done on as a global total last time? Now we would like to do regional.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional related info from Van Marle et al. 2017:
"Climate models should not use the CO2 emissions (or
nitrogen emissions if the nitrogen cycle is included in
the model
), as forcing because in general these emis-
sions are not net emissions to the atmosphere, but a
return pathway of previously sequestered carbon just as
respiration is
. The exceptions are CO2 emissions from
deforestation and peat fires. However, the models that do
not simulate land use change are recommended to use
land use change emissions prepared for AR6 (http://www.
mpimet.mpg.de/en/science/the-land-in-the-earth-system/
working-groups/climate-biogeosphere-interaction/
landuse-change-emission-data/)."

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably because it was only done on as a global total last time?

I think that's part of it. I also don't think CEDS did N2O last time, which is why we used PRIMAP. So yes, probably one to rethink.

Additional related info from Van Marle et al. 2017

Good find. We're going to have to ask Margreet or someone what this actually means, because it's not fully clear to me what this means for MAGICC (I think the answer is, don't include these for MAGICC either, but we should check).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, turned into issue #16

# unit_label = "Mt CO / yr"
# rcmip_gas = "CO"

# # TODO: check. I don't think we used this last time so we may want to ignore this again.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's right, but not yet sure I fully understand the reason.

Is it because it is assumed to be part of the global GCB value that we'll harmonize to?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it because it is assumed to be part of the global GCB value that we'll harmonize to?

I believe so (we should ask Thomas whether he thinks it's already part of the GCB value or not).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, turned into issue #17

@znichollscr
Copy link
Collaborator Author

Any tips/instructions on how to still complete the download? Or how to restart?

Ah yes, would be a good one to add to the README. I think you basically just run esgpull retry then esgpull download and it should restart...

@jkikstra
Copy link
Collaborator

Any tips/instructions on how to still complete the download? Or how to restart?

Ah yes, would be a good one to add to the README. I think you basically just run esgpull retry then esgpull download and it should restart...

Will add it to a README update PR later.

esgpull retry indeed looks like it did the trick, it's now restarted on my computer (not sure yet if it will be successful, but at least the downloads have started:

(esgf) C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical>esgpull retry
Sent back to the queue: 6 error

(esgf) C:\Users\kikstra\Documents\GitHub\emissions_harmonization_historical>esgpull download
<8b2ffd>
└── tags:        bb4cmip
    distrib:     False
    latest:      True
    replica:     None
    retracted:   False
    grid_label:  gn
    mip_era:     CMIP6Plus
    project:     input4MIPs
    source_id:   DRES-CMIP-BB4CMIP7-1-0
    variable_id: BC, CH4, CO, CO2, N2O, NH3, NOx, OC, SO2, gridcellarea
    files:       7.2 GiB / 11.1 GiB [13/19]
[0] <2c21a1>  12% ---- ----------------------------------- · 64.0/538.2 MiB · ? · esgf-data2.llnl.gov
[1] <40e871>   9% --- ------------------------------------ · 64.0/720.8 MiB · ? · esgf-data2.llnl.gov
⠸ 0/4 --:--

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants