Merge pull request #102 from bcdev/forman-19-write_levels

Write multi-level datasets

Author: forman

.coveragerc

@@ -0,0 +1,3 @@
+[run]
+omit =
+    zappend/contrib/*.py

CHANGES.md

@@ -1,3 +1,22 @@
+## Version 0.8.0 (in development)
+
+* Added module `zappend.contrib` that contributes functions to 
+  zappend's core functionality.
+
+* Added experimental function `zappend.contrib.write_levels()` that generates
+  datasets using the
+  [multi-level dataset format](https://xcube.readthedocs.io/en/latest/mldatasets.html)
+  as specified by
+  [xcube](https://github.com/xcube-dev/xcube).
+  It resembles the `store.write_data(cube, "<name>.levels", ...)` method
+  provided by the xcube filesystem data stores ("file", "s3", "memory", etc.).
+  The zappend version may be used for potentially very large datasets in terms 
+  of dimension sizes or for datasets with very large number of chunks.
+  It is considerably slower than the xcube version (which basically uses
+  `xarray.to_zarr()` for each resolution level), but should run robustly with 
+  stable memory consumption. 
+  The function requires `xcube` package to be installed. (#19)
+
 ## Version 0.7.1 (from 2024-05-30)
 
 * The function `zappend.api.zappend()` now returns the number of slices 

docs/api.md

@@ -45,3 +45,11 @@ All described objects can be imported from the `zappend.api` module.
 ::: zappend.api.ConfigLike
     options:
         show_root_heading: true
+
+## Contributions
+
+::: zappend.contrib
+
+### Function `write_levels()`
+
+::: zappend.contrib.write_levels

tests/contrib/__init__.py

@@ -0,0 +1,3 @@
+# Copyright © 2024 Norman Fomferra and contributors
+# Permissions are hereby granted under the terms of the MIT License:
+# https://opensource.org/licenses/MIT.