This repository contains the LEAP Catalog metadata for a collection of atmospheric datasets and model outputs prepared for CorrDiff-based climate downscaling experiments over the New York City region.
The collection includes:
- ERA5 atmospheric inputs interpolated to the HRRR grid
- HRRR reference datasets for the New York City region
- CorrDiff model inputs
- CorrDiff-generated outputs
- Reference target datasets
- CorrDiff model weights
The datasets are stored as NetCDF files on the Open Storage Network and are registered through the LEAP Data Catalog.
After this feedstock is registered and deployed, the public LEAP Catalog page will provide a single landing page for the full CorrDiff collection.
Catalog:
https://catalog.leap.columbia.edu/
The final CorrDiff catalog URL will be added here after deployment.
S3-compatible location:
s3://leap-pangeo-manual/hackathon-2026/corrdiff/
OSN endpoint:
https://nyu1.osn.mghpcc.org
Bucket:
leap-pangeo-manual
Prefix:
hackathon-2026/corrdiff/
These files contain ERA5 atmospheric fields interpolated to the 128-by-128 HRRR grid used for CorrDiff experiments.
Available years:
- 2020
- 2021
- 2022
- 2023
- 2024
- 2025
Filename pattern:
ERA5_hrrr_interp_128_<year>.nc
Examples:
ERA5_hrrr_interp_128_2020.nc
ERA5_hrrr_interp_128_2021.nc
ERA5_hrrr_interp_128_2025.nc
These files contain HRRR atmospheric reference data for the New York City region.
Available datasets:
hrrr_NYC_128_2020_2025.nc
hrrr_NYC_128_2021.nc
hrrr_NYC_128_2022.nc
hrrr_NYC_128_2023.nc
hrrr_NYC_128_2024.nc
hrrr_NYC_128_2025.nc
These files contain atmospheric inputs used in historical and recent-period CorrDiff experiments.
Available years:
- 1945
- 1955
- 1965
- 1975
- 1985
- 1995
- 2005
- 2020
- 2021
- 2025
Filename pattern:
input_<year>.nc
These files contain high-resolution atmospheric fields generated by CorrDiff.
Available years:
- 1945
- 1955
- 1965
- 1975
- 1985
- 1995
- 2005
- 2020
- 2021
- 2025
Filename pattern:
output_<year>.nc
These files contain reference target fields used to evaluate CorrDiff-generated outputs.
Available years:
- 1945
- 1955
- 1965
- 1975
- 1985
- 1995
- 2005
- 2020
- 2021
- 2025
Filename pattern:
target_<year>.nc
The collection also includes the following model artifacts:
corrdiff_weights/EDMPrecondSuperResolution.0.10000384.mdlus
corrdiff_weights/UNet.0.10000384.mdlus
These model-weight files are hosted in the same OSN collection but are not registered as NetCDF data stores in the LEAP Catalog.
The feedstock metadata is defined in:
feedstock/meta.yaml
The catalog entries are defined in:
feedstock/catalog.yaml
Each canonical NetCDF file is registered as an individual store under a single CorrDiff feedstock page.
The OSN directory also contains split 2021 files:
ERA5_hrrr_interp_128_2021_1.nc
ERA5_hrrr_interp_128_2021_2.nc
hrrr_NYC_128_2021_1.nc
hrrr_NYC_128_2021_2.nc
These files appear to be intermediate partitions of the complete 2021 datasets and are therefore not registered as primary catalog entries.
The canonical 2021 files are:
ERA5_hrrr_interp_128_2021.nc
hrrr_NYC_128_2021.nc
Apache License 2.0.
Sarika de Bruyn
GitHub: sarika-debruyn
EOF
Then inspect it:
```bash
cat README.md