test_nested[static_flat] fails on Debian sid Python 3.9

[avalentino]

Minimal, reproducible code sample, a copy-pastable example if possible

$ python3.9 -m pytest

============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-0.13.0
rootdir: /build/zarr-2.10.2+ds, configfile: pytest.ini
collected 2598 items

zarr/tests/test_attrs.py ........                                        [  0%]
zarr/tests/test_convenience.py ......................................... [  1%]
......................................                                   [  3%]
zarr/tests/test_core.py ................................................ [  5%]
........................................................................ [  7%]
....................................................................ssss [ 10%]
sssssssssssssssssssssssssssssssssssssssssss............................. [ 13%]
..................................................................ssssss [ 16%]
ssssssssssssssssssssssssssssssssssssssssss.............................. [ 19%]
.................sssssssssssssss.sssssssss.sssssssssssssssssssssssssssss [ 21%]
sssssss.sssssssss.ssssssssssssss.sssssssssssssssssssss.sssssssss.sssssss [ 24%]
ssssssssssssss.......................................................... [ 27%]
........................................................................ [ 30%]
........................................................................ [ 32%]
........................................................................ [ 35%]
........................................................................ [ 38%]
........................................................................ [ 41%]
.........sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 43%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 46%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssss              [ 49%]
zarr/tests/test_creation.py ..............                               [ 49%]
zarr/tests/test_dim_separator.py .........sssssssssssssss.........sssFx. [ 51%]
...x..sss                                                                [ 51%]
zarr/tests/test_filters.py .......                                       [ 51%]
zarr/tests/test_hierarchy.py ........................................... [ 53%]
....................sssssssssssssssssssss.....................ssssssssss [ 56%]
sssssssssssssssssssssssssssssssssss..................................... [ 58%]
.....ssssssssssssssssssssssssssssssssssssssssss......................... [ 61%]
............................................                             [ 63%]
zarr/tests/test_indexing.py .................................xx          [ 64%]
zarr/tests/test_info.py .                                                [ 64%]
zarr/tests/test_meta.py ................                                 [ 65%]
zarr/tests/test_storage.py ............s........................s....... [ 67%]
.................s...........................................sssssssssss [ 69%]
ssssssssssssssssssssssssssssssssssssssssssss............................ [ 72%]
....................................ssssssssssssssssssssssssssssssssssss [ 75%]
ssssssssssssssssssssssssssss.............s.............................. [ 78%]
...........................................................sssssssssssss [ 80%]
ssssssssssss.........................sssssssssssssssssssssssssssssssssss [ 83%]
sssssssssssssss..................................................sssssss [ 86%]
sssssssssssssssssssssssssssssssssssssssss.............s................s [ 89%]
sssssssssssssssssssssssss..                                              [ 90%]
zarr/tests/test_sync.py ................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 97%]
..............................................                           [ 99%]
zarr/tests/test_util.py ..............                                   [100%]

=================================== FAILURES ===================================
___________________________ test_nested[static_flat] ___________________________

dataset = '/build/zarr-2.10.2+ds/.pybuild/cpython3_3.9_zarr/build/fixture/flat'

    def test_nested(dataset):
        """
        Use NestedDirectoryStore to open the dataset fixture. This is the only
        method that is expected to successfully open legacy nested datasets
        without the dimension_separator metadata. However, for none-Nested
        datasets without any metadata, NestedDirectoryStore will fail.
        """
        failure = (
            "flat_legacy" in dataset or
            "directory_default" in dataset or
            "fs_default" in dataset
        )
>       verify(Array(store=NestedDirectoryStore(dataset)), failure)

zarr/tests/test_dim_separator.py:129:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

array = <zarr.core.Array (2, 2) int64>, expect_failure = False

    def verify(array, expect_failure=False):
        try:
>           assert_array_equal(array[:], [[1, 2], [3, 4]])
E           AssertionError:
E           Arrays are not equal
E
E           Mismatched elements: 4 / 4 (100%)
E           Max absolute difference: 4
E           Max relative difference: 1.
E            x: array([[0, 0],
E                  [0, 0]])
E            y: array([[1, 2],
E                  [3, 4]])

zarr/tests/test_dim_separator.py:78: AssertionError
=============================== warnings summary ===============================
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py::test_open_array
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite_group
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_init_array
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_init_array_path
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite_path
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_init_array_overwrite_chunk_store
.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::TestN5Store::test_filters
  /build/zarr-2.10.2+ds/.pybuild/cpython3_3.9_zarr/build/zarr/n5.py:708: RuntimeWarning: Not all N5 implementations support blosc compression (yet). You might not be able to open the dataset with another N5 library.
    warnings.warn(

.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py::test_compression_args
  /build/zarr-2.10.2+ds/.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py:472: UserWarning: 'compression' keyword argument overridden by 'compressor'
    z = create(100, compressor=Zlib(9), compression='bz2', compression_opts=1)

.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py::test_compression_args
  /build/zarr-2.10.2+ds/.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py:472: UserWarning: 'compression_opts' keyword argument overridden by 'compressor'
    z = create(100, compressor=Zlib(9), compression='bz2', compression_opts=1)

.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py::test_compression_args
  /build/zarr-2.10.2+ds/.pybuild/cpython3_3.9_zarr/build/zarr/tests/test_creation.py:478: UserWarning: 'compression_opts' keyword argument overridden by 'compressor'
    z = create(100, compressor=Zlib(9), compression_opts=1)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
============================= slowest 10 durations =============================
25.34s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_indexing.py::test_set_orthogonal_selection_3d
6.08s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_storage.py::test_format_compatibility
3.94s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_indexing.py::test_set_orthogonal_selection_2d
2.10s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_core.py::TestArrayWithNestedDirectoryStore::test_object_arrays_vlen_array
2.02s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_indexing.py::test_get_orthogonal_selection_3d
1.80s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_core.py::TestArrayWithBZ2Compressor::test_object_arrays_vlen_array
1.77s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_core.py::TestArrayWithDirectoryStore::test_object_arrays_vlen_array
1.72s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_sync.py::TestArrayWithThreadSynchronizer::test_object_arrays_vlen_array
1.66s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_core.py::TestArrayWithSQLiteStore::test_object_arrays_vlen_array
1.65s call     .pybuild/cpython3_3.9_zarr/build/zarr/tests/test_core.py::TestArrayWithDBMStore::test_object_arrays_vlen_array
=========================== short test summary info ============================
FAILED zarr/tests/test_dim_separator.py::test_nested[static_flat] - Assertion...
= 1 failed, 1768 passed, 825 skipped, 4 xfailed, 11 warnings in 148.25s (0:02:28) =

Problem description

The "test_nested[static_flat]" test fails in the current Debian sid with Python 3.9.
Of course one would expect that all automatic tests pass.

Version and installation information

• Value of zarr.__version__ 2.10.2
• Value of numcodecs.__version__ 0.8.1
• Version of Python interpreter 3.9.7
• Operating system (Linux/Windows/Mac) GNU/Linux Debian Sid (22/10/2021)
• How Zarr was installed (e.g., "using pip into virtual environment", or "using conda"): setup.py during the deb package build

[joshmoore]

This matches the failure in conda-forge/zarr-feedstock#56. Having this pinpointed outside of conda-forge is useful. Thanks, @avalentino.

[jakirkham]

Agreed thank you! 😄

Does this only happen with Python 3.9? What happens on other Python versions (like 3.8)?

[joshmoore]

Activating 3.9 doesn't seem to show the same error. I'll try to reproduce in a Sid docker.

[joshmoore]

With the following docker file:

# See: https://github.com/zarr-developers/zarr-python/issues/858
FROM debian:sid
RUN apt-get update -y \
 && apt-get install -y \
    git \
    python3 \
    python3-setuptools \
    python3-pip \
    python3-pytest \
    python3-msgpack \
    python3-fsspec \
    python3-numcodecs \
    python3-asciitree \
    python3-fasteners

WORKDIR /tmp
RUN git clone -b v2.10.2 git://github.com/zarr-developers/zarr-python
WORKDIR /tmp/zarr-python/

RUN python3 setup.py build
#RUN python3 -X tracemalloc setup.py test
ENTRYPOINT ["python3", "-m", "pytest"]

I get:

(base) ~/tmp/sid $docker run --rm -ti sid  \-svxk "test_dim_separator and static_flat"
=============================================================================================================== test session starts ================================================================================================================
platform linux -- Python 3.9.8, pytest-6.2.5, py-1.10.0, pluggy-0.13.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /tmp/zarr-python, configfile: pytest.ini
collected 2598 items / 2590 deselected / 8 selected

zarr/tests/test_dim_separator.py::test_open[static_flat] PASSED
zarr/tests/test_dim_separator.py::test_open[static_flat_legacy] PASSED
zarr/tests/test_dim_separator.py::test_fsstore[static_flat] PASSED
zarr/tests/test_dim_separator.py::test_fsstore[static_flat_legacy] PASSED
zarr/tests/test_dim_separator.py::test_directory[static_flat] PASSED
zarr/tests/test_dim_separator.py::test_directory[static_flat_legacy] PASSED
zarr/tests/test_dim_separator.py::test_nested[static_flat] PASSED
zarr/tests/test_dim_separator.py::test_nested[static_flat_legacy] XFAIL

=============================================================================================================== slowest 10 durations ===============================================================================================================

(10 durations < 0.005s hidden.  Use -vv to show these durations.)
================================================================================================== 7 passed, 2590 deselected, 1 xfailed in 5.00s ===================================================================================================

(and more generally all enabled tests pass). Elided listing of installed libraries:

dpkg -l
...
ii  python3-asciitree          0.3.3-3                        all          Draw tree structures using (ASCII or Unicode) characters
ii  python3-fasteners          0.14.1-2                       all          provides useful locks - Python 3.x
ii  python3-fsspec             0.8.4-1                        all          specification that Python filesystems should adhere to (Python 3)
ii  python3-msgpack            1.0.2-2                        amd64        Python 3 implementation of MessagePack format
ii  python3-numcodecs          0.8.1+ds-3                     amd64        buffer compression and transformation codecs for Python
ii  python3-numpy              1:1.19.5-1                     amd64        Fast array facility to the Python 3 language
ii  python3.9                  3.9.8-2                        amd64        Interactive high-level object-oriented language (version 3.9)

@avalentino: can you help me modify the Dockerfile such that it will more closely match what you are seeing? I didn't see any patches that should have any effect.

[joshmoore]

Ok. I think I've finally tracked down the cause. My guess is that the fixtures/ directory was not available for testing and therefore they were being re-created and that incorrectly. Hopefully #871 should correct the situation.