DoctestPlus and optional dependencies

[flying-sheep]

First, similar to #59, there’s another doctest plugin that has its own set of features.

The only current feature of it I need is doctest_subpackage_requires. It’ll prevent descending into these subdirectories if (and only if) the listed dependencies are not installed.

This is a much better solution than just skipping stuff outright.

E.g.:

doctest_subpackage_requires = [
  "src/fast_array_utils/conv/scipy/* = scipy",
  "src/fast_array_utils/conv/scipy/_to_dense.py = numba",
  "src/fast_array_utils/stats/* = numba",
  "src/fast_array_utils/_plugins/dask.py = dask",
  "src/fast_array_utils/_plugins/numba_sparse.py = numba;scipy",
]

[ev-br]

This feature looks interesting!

A less sophisticated version exists as pytest_extra_ignore,

scipy_doctest/scipy_doctest/plugin.py

Line 57 in e1a9521

for entry in config.dt_config.pytest_extra_ignore:

(cf https://github.com/scipy/scipy/blob/main/scipy/conftest.py#L606 for an example usage).

The missing bit is the ability to accept the package/version, in the "path = requirement" syntax. Question: in your example, would a single "package" entry suffice, or you're looking for a full pyproject.toml specification, e.g. "scipy<=1.15;numpy>2"?
If the latter, I'm not immediately sure what the canonic way to parse these strings is, there surely is a reference implementation?

P.S. We are aware of pytest-doctestplus, cf #188 for instance.

[flying-sheep]

As long as both extensions work together, it’s fine. I thought at first they don’t but it was just a bug of scipy_doctest (#198)

If the latter, I'm not immediately sure what the canonic way to parse these strings is, there surely is a reference implementation?

You would use https://packaging.pypa.io/en/stable/ and https://docs.python.org/3/library/importlib.metadata.html#importlib.metadata.version

from importlib.metadata import version
from packaging.specifiers import Requirement

req = Requirement("numpy >=2, !=2.0.0")

if version(req.name) in req.specifier:
    do_thing()

[ev-br]

a bug of scipy_doctest (#198)

It's a feature, not a bug :-). Let's keep it in gh-198 though.

As long as both extensions work together

They do, to a degree. There's some discussion in a discord thread, https://discord.com/channels/786703927705862175/1020054666433802351

You would use https://packaging.pypa.io/en/stable/ and https://docs.python.org/3/library/importlib.metadata.html#importlib.metadata.version

Great, thank you! That's not the full story though? This does not seem to work for dev builds/installs?

$ python dev.py ipython
....
In [1]: from importlib.metadata import version

In [2]: version('scipy')

...

PackageNotFoundError: No package metadata was found for scipy

Works for a version installed from PyPI though, so is it a scipy-specific thing, or is it a meson-python specific thing, if you have any ideas?

[flying-sheep]

OK, we’re getting deep in the weeds. In short (using pillow as an example for reasons that will become immediately clear):

• importlib.metadata.version("pillow") is the standard way to get a version for a (distribution) package
• PIL.__version__ is a convention, not a standard, to get a version from a (python) package

Dependencies are specified in terms of distribution packages, so you’d add a dependency on pillow, not PIL, to get access to import PIL.

This does not seem to work for dev builds/installs?

It totally does. Standards (PEP 660) based approaches to install a package in editable mode (such as pip install -e .) will create package metadata in addition to making the imports available. I don‘t know what dev.py is, but it seems like it’s using an outdated way to do editable installs.

So until that’s fixed, it probably makes little sense to refer to dependencies as you can’t rely on them being installed correctly (i.e. with their metadata intact).

---

Feel free to check out session-info2, a package that relies completely on standard metadata to show installed packages in an env.

Ideally that would work in your dev envs.