HUGE list of dependencies - accident or intention?

[hoffch]

I am using xdoctest 1.1.3 installed with poetry. I just realized by chance that the list of xdoctest's dependencies is surprisingly large:

 - attrs >=19.2.0
 - colorama >=0.4.1
 - debugpy >=1.6.0
 - ipykernel >=6.11.0
 - IPython >=7.23.1
 - ipython-genutils >=0.2.0
 - jedi >=0.16
 - jinja2 >=3.0.0
 - jupyter-client >=7.0.0
 - jupyter-core >=4.7.0
 - nbconvert >=6.1.0
 - pyflakes >=2.2.0
 - Pygments >=2.4.1
 - pytest >=6.2.5
 - pytest-cov >=3.0.0
 - tomli >=0.2.0

With transitive dependencies, this sums up to 41 packages that are required by xdoctest. While I really like xdoctest, this actually prevents me from using it anymore. Too much complexity for the provided functionality. Plus the dependency installation slows down the CI unnecessarily.

Is that huge list really required? Or did some dev dependency (e.g. pyflakes) accidentally sneak into the main deps?

EDIT:
It seems a lot of dependencies are related to doctests in Jupyter notebooks. Maybe Jupyter support can be made an opt-in feature on installation using extras (pip install xdoctest[jupyter]). That would drastically reduce the dependencies for everyone not working with Jupyter notebooks.

[Erotemic]

There are no runtime dependencies for xdoctest. Poetry is being too aggressive extracting optional dependencies.

All requirements are separated by type here: https://github.com/Erotemic/xdoctest/tree/main/requirements Notice that runtime.txt is empty.

The reason I'm still on setuptools is because I parse these requirement.txt files and will optionally replace a ">=" with a "==" to pin to strict versions on user request. The extra_requires are defined here: https://github.com/Erotemic/xdoctest/blob/main/setup.py#L208 but the relevent code is:

    setupkw["extras_require"] = {
        "all": parse_requirements("requirements.txt", versions="loose"),
        "tests": parse_requirements("requirements/tests.txt", versions="loose"),
        "optional": parse_requirements("requirements/optional.txt", versions="loose"),
        "all-strict": parse_requirements("requirements.txt", versions="strict"),
        "runtime-strict": parse_requirements(
            "requirements/runtime.txt", versions="strict"
        ),
        "tests-strict": parse_requirements("requirements/tests.txt", versions="strict"),
        "optional-strict": parse_requirements(
            "requirements/optional.txt", versions="strict"
        ),
        'tests-binary': parse_requirements('requirements/tests-binary.txt'),
        'tests-binary-strict': parse_requirements('requirements/tests-binary.txt', versions='strict'),
        'colors': parse_requirements('requirements/colors.txt'),
        'jupyter': parse_requirements('requirements/jupyter.txt'),
    }

[Erotemic]

Is this an issue in the use of xdoctest with poetry? Will including xdoctest as a dependency bring all of these optional dependencies into the poetry lock file? If so can someone provide a minimal pyproject.toml that demonstrates this? If this is a problem, I'd like to either fix it here (or better in poetry because I don't think xdoctest is doing anything that is incorrect, and poetry shouldn't be including these dependencies unless you depend on something like xdoctest[all]. ).

[NellyWhads]

I can confirm this is a poetry configuration issue. When installing with uv, I get the following uv tree output

With "xdoctest>=1.2" in the dev group:

...
└── xdoctest v1.2.0 (group: dev)
...

With "xdoctest[all]>=1.2" in the dev group:

...
└── xdoctest[all] v1.2.0 (group: dev)
    ├── attrs v23.2.0 (extra: all)
    ├── debugpy v1.8.11 (extra: all)
    ├── ipykernel v6.29.5 (extra: all) (*)
    ├── ipython v8.31.0 (extra: all) (*)
    ├── ipython-genutils v0.2.0 (extra: all)
    ├── jedi v0.19.2 (extra: all) (*)
    ├── jinja2 v3.1.5 (extra: all) (*)
    ├── jupyter-client v7.4.9 (extra: all) (*)
    ├── jupyter-core v5.7.2 (extra: all) (*)
    ├── nbconvert v7.16.4 (extra: all) (*)
    ├── pyflakes v3.2.0 (extra: all)
    ├── pygments v2.18.0 (extra: all)
    ├── pytest v8.3.4 (extra: all) (*)
    ├── pytest-cov v5.0.0 (extra: all) (*)
    └── tomli v2.2.1 (extra: all)
...

[Erotemic]

I'm wondering if there is anything that can be done in this repo. Its annoying that pypistats also lists a bunch of "required" dependencies when really there are none.

Maybe this is because there is a top level requirements.txt that does point to all of the dependencies? Of course, the "install_requires" section of the setup.py file only points to requirements/runtime.txt, which is empty as there are no required dependencies. Everything else is in the "extra_requires" dictionary, which as far as I understand should never be considered as required.

My thought is that it would be nice for devs if requirements.txt included everything needed for a full test of the system, but if it is causing problems it could probably be removed. That being said, I don't know if it is causing problems.

[NellyWhads]

I can add that using hatchling/poetry-core to build packages has circumvented this issue for me in the past.