Skip to content

Commit

Permalink
Add basic handling of GFED4 BB4CMIP data
Browse files Browse the repository at this point in the history
  • Loading branch information
znichollscr committed Oct 17, 2024
1 parent 2e600a5 commit 4bdd2a2
Show file tree
Hide file tree
Showing 5 changed files with 139 additions and 1 deletion.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Processed data
data/national/ceds/processed
data/national/gfed/processed
data/national/gfed-bb4cmip/processed

# CEDS data
data/national/ceds/data_raw

# GFED data
data/national/gfed/data_raw
data/national/gfed-bb4cmip/data_raw

# pixi stuff
.pixi
Expand Down
51 changes: 51 additions & 0 deletions data/national/gfed-bb4cmip/data_raw/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
Biomass burning data is on the ESGF here: https://aims2.llnl.gov/search?project=input4MIPs&activeFacets=%7B%22mip_era%22%3A%22CMIP6Plus%22%2C%22institution_id%22%3A%22DRES%22%7D

Download the data of interest here.
This can be done with the following commands
and [esgpull](https://esgf.github.io/esgf-download/)
(if you're not on a unix-based system,
create a virtual environment using whatever environment manager you like
then remove `venv/bin/` from all the commands below).

```sh
# Install esgpull
# (see note above for non-unix systems)
python3 -m venv venv
venv/bin/pip install esgpull
venv/bin/esgpull self install

# In the diaglog that pops up, install here
# to ensure that the paths in the notebooks behave.
venv/bin/esgpull config api.index_node esgf-node.llnl.gov
```

Then, check that the installation worked correctly by running

```sh
venv/bin/esgpull config
```

The output should look something like

```
─────────────────── /path/to/emissions_harmonization_historical/data/national/gfed-bb4cmip/data_raw/config.toml ───────────────────
[paths]
...
data = "/path/to/emissions_harmonization_historical/data/national/gfed-bb4cmip/data_raw/data"
...
[api]
index_node = "esgf-node.llnl.gov"
...
```

Then, download the data

```sh
# Comma-separated list
# You can search with esgpull search e.g. `venv/bin/esgpull search --all project:input4MIPs mip_era:CMIP6Plus source_id:DRES-CMIP-BB4CMIP7-1-0 grid_label:gn`
variable_ids_to_grab="BC,CH4,CO,CO2,N2O,NH3,NOx,OC,SO2,gridcellarea"
venv/bin/esgpull add --tag bb4cmip --track variable_id:"${variable_ids_to_grab}" project:input4MIPs mip_era:CMIP6Plus source_id:DRES-CMIP-BB4CMIP7-1-0 grid_label:gn
venv/bin/esgpull update -y --tag bb4cmip
venv/bin/esgpull download
```
59 changes: 59 additions & 0 deletions notebooks/0103_GFED4-BB4CMIP-prepare.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# ---
# jupyter:
# jupytext:
# text_representation:
# extension: .py
# format_name: percent
# format_version: '1.3'
# jupytext_version: 1.16.4
# kernelspec:
# display_name: Python 3 (ipykernel)
# language: python
# name: python3
# ---

# %%
from collections import defaultdict

import xarray as xr

from emissions_harmonization_historical.constants import DATA_ROOT

# %%
raw_data_path = DATA_ROOT / "national/gfed-bb4cmip/data_raw/data"
raw_data_path

# %% [markdown]
# Group raw data into variable groups which can be used for processing.

# %%
bb4cmip_files = list(raw_data_path.rglob("*.nc"))
bb4cmip_file_groups = defaultdict(list)

for file in bb4cmip_files:
variable = file.name.split("_")[0]
bb4cmip_file_groups[variable].append(file)

bb4cmip_file_groups.keys()

# %% [markdown]
# An example of loading data.

# %%
cell_area = xr.open_mfdataset(bb4cmip_file_groups["gridcellarea"])
cell_area

# %%
gas = "BC"
gas_ds = xr.open_mfdataset(bb4cmip_file_groups["BC"])
gas_ds

# %%
global_sum_emissions = (gas_ds["BC"] * cell_area["gridcellarea"]).sum(["latitude", "longitude"]).compute()
global_sum_emissions

# %%
global_sum_emissions.plot()

# %%
global_sum_emissions.groupby("time.year").mean().plot()
27 changes: 26 additions & 1 deletion pixi.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ h5py = ">=3.11.0,<4"
netcdf4 = ">=1.7.1,<2"
h5netcdf = ">=1.4.0,<2"
pooch = ">=1.8.2,<2"
nc-time-axis = ">=1.4.1,<2"

[tool.pixi.pypi-dependencies]
emissions_harmonization_historical = { path = ".", editable = true }
Expand Down

0 comments on commit 4bdd2a2

Please sign in to comment.