Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for NSIDC SMAP gridded data - L3, L4, L2-Gridded #35

Open
5 tasks
D-Auty opened this issue Dec 31, 2024 · 1 comment
Open
5 tasks

Add Support for NSIDC SMAP gridded data - L3, L4, L2-Gridded #35

D-Auty opened this issue Dec 31, 2024 · 1 comment

Comments

@D-Auty
Copy link

D-Auty commented Dec 31, 2024

NSIDC SMAP Datasets include:

  • SPL4CMDL, SPL4SMAU, SPL4SMGP, SPL4SMLM
  • SPl3SMA, SPL3SMAP; SPL3SMP, SPL3SMP_E; SPL3FTA; SPL3FTP, SPL3FTP_E; SPL2SMSP (tiled, gridded)

Issues with Net2Cog appear to be:

  • Error "expected str, bytes or os.PathLike object, not list" for harmony request #34 Processing failures lead to an empty list of output files, which leads to the misleading error reported on termination
  • Add support for grouped datasets #37 Variable Hierarchies are not handled correctly - xarray does not automatically load nested variables.
    • Upgrade Xarray and related library versions
    • Use Xarray.datatree vs. Xarray.dataset, and convert variable selection back to dataset, or work with DataArray directly.
    • (I suspect code should be restructured to handle multi-variables and variable hierarchies together, see below)
  • Net2Cog should not default to Web Mercator projection if CRS is not explicitly set. #12 Related to above - CRS & Dimensions should be handled on a per-variable case, per-variable references.
    • e.g., SMAP L3 has xdim and ydim dimension variable names
    • Upgraded xarray and rioxarray/rasterio appear to handle CF conventions well - including SMAP
      • No need to implement add_crs after to_raster in all cases
      • Code currently assumes geographic grid and adds EPSG:4326 CRS
        • this should be special case - only if nc_xarray[var].rio.crs is None
        • and perhaps should have additional logic to verify geographic grid use-cases
          • Dimension standard names are latitude/longitude, and/or units == degrees
    • This will support multiple CRS per data file, use-cases not currently supported (required for SMAP and ICESat-2)
  • For generality (but not necessary for SMAP)
    • Dimensions variables must be in data hierarchy, which Datatree honors (inheritance)
but grid-mapping variables may be “up-and-over”, which requires interpreting absolute or possibly relative path references in the CF grid-mapping attribute.
    • RioXarray may already handle this correctly - TBD
    • Other datasets to support (TBD, following is set of collections associated in UAT)
      • TEMPO
      • PREFIRE
      • CERES_EBAF,
      • CPF_CERES_N20_INTERCAL,
      • CLARREO_SIMTEST_L1A
  • Add support for multiple variables #32 Support for Multi-variable or "all" variables ignoring the non-spatial, incompatible variables
    • Reporting conversion successes vs failures is TBD, but may or may not be relevant.
@frankinspace
Copy link
Member

@D-Auty

Dimensions variables must be in data hierarchy ...

I don't quite understand the issue here. Is this a speculation that some datasets that use grid-mapping variables might not be supported? If so I think we should split this out from the SMAP specific stuff and hopefully find a concrete use case to build around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🆕 New
Development

No branches or pull requests

2 participants