refactor warnings #3098

d-v-b · 2025-05-25T19:45:40Z

this PR adds zarr-specific subclasses of FutureWarning and DeprecationWarning, ~~so that we can expose routines for silencing future / deprecation warnings specifically emitted by zarr-python.~~

~~This PR is a draft until I add the silencing routines later in this PR.~~

dstansby · 2025-05-31T07:58:44Z

👍 to having our own zarr warning sub-classes, I think I'm 👎 on having some of our own warning silencing helpers, but depends on what the proposal is I guess. So would be good to have separate PRs for those.

…factor-warnings

d-v-b · 2025-06-30T08:37:02Z

I amended the PR description to remove the goal of providing top-level functions for users to silence zarr warnings. For now, users can use the built-in warnings library to control warnings.

codecov · 2025-07-11T18:40:19Z

Codecov Report

❌ Patch coverage is 14.54545% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.74%. Comparing base (e410173) to head (713bda9).

Files with missing lines	Patch %	Lines
src/zarr/api/asynchronous.py	22.22%	14 Missing ⚠️
src/zarr/core/group.py	0.00%	7 Missing ⚠️
src/zarr/core/array.py	16.66%	5 Missing ⚠️
src/zarr/errors.py	0.00%	5 Missing ⚠️
src/zarr/api/synchronous.py	0.00%	2 Missing ⚠️
src/zarr/testing/__init__.py	0.00%	2 Missing ⚠️
src/zarr/_compat.py	0.00%	1 Missing ⚠️
src/zarr/convenience.py	0.00%	1 Missing ⚠️
src/zarr/core/buffer/gpu.py	0.00%	1 Missing ⚠️
src/zarr/core/chunk_grids.py	0.00%	1 Missing ⚠️
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3098      +/-   ##
==========================================
+ Coverage   60.72%   60.74%   +0.02%     
==========================================
  Files          78       78              
  Lines        9408     9426      +18     
==========================================
+ Hits         5713     5726      +13     
- Misses       3695     3700       +5

Files with missing lines	Coverage Δ
src/zarr/codecs/transpose.py	`45.61% <ø> (ø)`
src/zarr/_compat.py	`0.00% <0.00%> (ø)`
src/zarr/convenience.py	`0.00% <0.00%> (ø)`
src/zarr/core/buffer/gpu.py	`41.25% <0.00%> (-0.53%)`	⬇️
src/zarr/core/chunk_grids.py	`59.81% <0.00%> (-0.57%)`	⬇️
src/zarr/core/codec_pipeline.py	`70.09% <0.00%> (-1.34%)`	⬇️
src/zarr/core/common.py	`48.76% <66.66%> (-0.41%)`	⬇️
src/zarr/core/dtype/common.py	`27.71% <0.00%> (ø)`
src/zarr/core/metadata/v2.py	`58.13% <50.00%> (-0.34%)`	⬇️
src/zarr/creation.py	`0.00% <0.00%> (ø)`
... and 9 more

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…into refactor-warnings

d-v-b · 2025-07-31T22:00:16Z

this will allow users who are tired of seeing zarr-specific warnings to suppress them.

dstansby · 2025-08-01T08:38:11Z

I milestoned as 3.2.0, but I guess this isn't breaking (because the new warnings subclass the old warnings), so could go in 3.1.2 instead?

dstansby

Code changes look good overall, but:

Tests need fixing
I think there's more warnings in the list of our warning filters that should be converted to the new zarr-specific warnings
The new warning classes shold be included in the API docs

pyproject.toml

src/zarr/errors.py

tests/test_api.py

d-v-b · 2025-08-01T12:56:21Z

I removed almost all the ignored warnings from pyproject.toml, at the cost of inserting a bunch of explicit with pytest.warns() statements throughout our tests. This was useful because it revealed how many warnings zarr emits during normal usage. For consolidated metadata in particular I think we need to rethink emitting a warning every time its used. I don't think this feature is typically used by individual users but rather libraries like xarray, in which case the best way to convey the off-spec-ness of consolidated metadata is via documentation.

d-v-b · 2025-08-01T12:57:00Z

we should also consider changing our ZipStore implementation to hide the UserWarning emitted whenever a key is modified.

TomAugspurger · 2025-08-01T13:09:02Z

removed almost all the ignored warnings from pyproject.toml, at the cost of inserting a bunch of explicit with pytest.warns() statements throughout our tests

Thanks for doing this. I was going to suggest that, but didn't want to assign work to you that's only tangentially related to the PR. I think that being explicit in the tests about where warnings are raised is a good practice.

d-v-b · 2025-08-01T13:11:26Z

consistent with my "test code models user code" theory, excluding every warning individually from the tests has given me a newfound appreciation for all the warnings zarr-python emits on a regular basis

into refactor-warnings

d-v-b · 2025-08-01T14:01:36Z

I don't think the coverage report is very useful here. If we discount it, I think this is ready to go

dstansby · 2025-08-01T15:10:09Z

docs/user-guide/consolidated_metadata.rst

@@ -27,6 +27,8 @@ In Python, the consolidated metadata is available on the ``.consolidated_metadat
 attribute of the ``GroupMetadata`` object.

   >>> import zarr
+   >>> import warnings
+   >>> warnings.filterwarnings("ignore", category=UserWarning)


Is it possible to remove these new lines, and instead include the warnings in the docstest output? Instead of hiding the warnings, it would be nice to be honest about them 😄

dstansby · 2025-08-01T15:11:59Z

src/zarr/errors.py

+
+class ZarrFutureWarning(FutureWarning):
+    """
+    A warning raised to indicate when a construct will change semantically in the future.


What does "construct will change semantically" mean (certainly I don't know!)? Can this be rewritten in less jargon-y language?

dstansby · 2025-08-01T15:12:14Z

src/zarr/errors.py

+
+class ZarrDeprecationWarning(DeprecationWarning):
+    """
+    A warning raised to indicate that a construct will be removed in a future release.


Suggested change

A warning raised to indicate that a construct will be removed in a future release.

A warning raised to indicate that a feature will be removed in a future release.

dstansby · 2025-08-01T15:13:11Z

src/zarr/errors.py

+
+class ZarrRuntimeWarning(RuntimeWarning):
+    """
+    A warning for dubious runtime behavior.


What's the difference between a runtime warning and a user warning? It's not clear to me from these descriptions how to choose between them when introducing a new warning.

dstansby · 2025-08-01T15:14:56Z

tests/test_codecs/test_codecs.py

+        (BytesCodec(), TransposeCodec(order=order_from_dim("F", 2))),
+        (TransposeCodec(order=order_from_dim("F", 2)),),
+    ],
+)


Why the change in paramterization here? Could the store parameterization go back as well as the new codec paraetrization?

dstansby · 2025-08-01T22:02:44Z

changes/3098.misc.rst

+Define Zarr-specific warning classes ``ZarrDeprecationWarning`` and ``ZarrFutureWarning``, that
+subclass ``DeprecationWarning`` and ``FutureWarning``, respectively.


Suggested change

Define Zarr-specific warning classes ``ZarrDeprecationWarning`` and ``ZarrFutureWarning``, that

subclass ``DeprecationWarning`` and ``FutureWarning``, respectively.

Define Zarr-specific warning classes `ZarrDeprecationWarning` and `ZarrFutureWarning`, that

subclass `DeprecationWarning` and `FutureWarning`, respectively.

Because these are in the API docs, use single quotes to link to them.

define zarr-specific FutureWarning and DeprecationWarning

79dd9e2

github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label May 25, 2025

Merge branch 'main' of github.com:zarr-developers/zarr-python into re…

b2e25cf

…factor-warnings

make unstablespecificationwarning an instance of zarrfuturewarning

7c644eb

d-v-b marked this pull request as ready for review June 30, 2025 08:48

d-v-b requested review from jhamman and TomAugspurger June 30, 2025 08:48

d-v-b added this to the 3.0.9 milestone Jun 30, 2025

use pytest.warns instead of pytest.raises

c604075

d-v-b removed this from the 3.0.9 milestone Jun 30, 2025

Merge branch 'main' into refactor-warnings

b964ff6

d-v-b added 2 commits July 11, 2025 20:41

Merge branch 'main' into refactor-warnings

12bea9c

Merge branch 'main' into refactor-warnings

8267c0c

dstansby added this to the 3.2.0 milestone Jul 31, 2025

d-v-b added 2 commits July 31, 2025 23:56

Merge branch 'main' of https://github.com/zarr-developers/zarr-python …

702ef09

…into refactor-warnings

changelog

64e1405

github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Jul 31, 2025

d-v-b requested a review from a team July 31, 2025 21:59

dstansby requested changes Aug 1, 2025

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

src/zarr/errors.py Outdated Show resolved Hide resolved

tests/test_api.py Outdated Show resolved Hide resolved

d-v-b added 4 commits August 1, 2025 10:56

ensure that all deprecations are ZarrDeprecations

d322cad

add docstrings and export warnings

31e63aa

fix imports

917fd3d

handle warnings in tests explicitly; make userwarnings ZarrUserWarning

9055d3b

d-v-b added 2 commits August 1, 2025 15:03

fix doctests by making them more realistic

fee48d3

lint and fix typo

6a208eb

d-v-b added 6 commits August 1, 2025 15:11

Merge branch 'main' into refactor-warnings

8a2bb9e

move unstable spec warning to errors

a235660

Merge branch 'refactor-warnings' of https://github.com/d-v-b/zarr-python

8afdce9

into refactor-warnings

handle warnings in gpu tests

33a9184

handle more warnings

fc72752

handle another warning

42b4cb5

d-v-b requested a review from dstansby August 1, 2025 14:01

Fix exports

713bda9

dstansby reviewed Aug 1, 2025

View reviewed changes

	A warning raised to indicate that a construct will be removed in a future release.
	A warning raised to indicate that a feature will be removed in a future release.

		Define Zarr-specific warning classes ``ZarrDeprecationWarning`` and ``ZarrFutureWarning``, that
		subclass ``DeprecationWarning`` and ``FutureWarning``, respectively.

Uh oh!

refactor warnings #3098

Are you sure you want to change the base?

refactor warnings #3098

Conversation

d-v-b commented May 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dstansby commented May 31, 2025

Uh oh!

d-v-b commented Jun 30, 2025

Uh oh!

codecov bot commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

d-v-b commented Jul 31, 2025

Uh oh!

dstansby commented Aug 1, 2025

Uh oh!

dstansby left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

d-v-b commented Aug 1, 2025

Uh oh!

d-v-b commented Aug 1, 2025

Uh oh!

TomAugspurger commented Aug 1, 2025

Uh oh!

d-v-b commented Aug 1, 2025

Uh oh!

d-v-b commented Aug 1, 2025

Uh oh!

dstansby Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

dstansby Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

dstansby Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

dstansby Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

dstansby Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

dstansby Aug 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

d-v-b commented May 25, 2025 •

edited

Loading

codecov bot commented Jul 11, 2025 •

edited

Loading