Optional dependency handling?

[bollwyvl]

Comment:

Thanks for maintaining this!

Downstream on conda-forge/dagster-libs-feedstock#12, I've found a dependency on polars[pyarrow]. For now, I guess I'll manually add the pin to >=7.0.0, but this seems unsatisfying (ha!).

Looking upstream, it looks like there is quite the stable of optional integrations:

adbc = ["adbc_driver_sqlite"]
cloudpickle = ["cloudpickle"]
connectorx = ["connectorx >= 0.3.2"]
deltalake = ["deltalake >= 0.13.0"]
fsspec = ["fsspec"]
gevent = ["gevent"]
matplotlib = ["matplotlib"]
numpy = ["numpy >= 1.16.0"]
openpyxl = ["openpyxl >= 3.0.0"]
pandas = ["pyarrow >= 7.0.0", "pandas"]
pyarrow = ["pyarrow >= 7.0.0"]
pydantic = ["pydantic"]
pyiceberg = ["pyiceberg >= 0.5.0"]
pyxlsb = ["pyxlsb >= 1.0"]
sqlalchemy = ["sqlalchemy", "pandas"]
timezone = ["backports.zoneinfo; python_version < '3.9'", "tzdata; platform_system == 'Windows'"]
xlsx2csv = ["xlsx2csv >= 0.8.0"]
xlsxwriter = ["xlsxwriter"]

Would it be possible to do one of the following:

• add all the versioned ones as run_constrained
  • lightweight, and not very hard for downstreams to reason about
• add noarch outputs for all the [extra] deps, e.g. polars-with-pyarrow
  • better for import-based testing, but still not going to tease out when version pins change
  • this could even be in a separate repo, so it's only one build
• ???

Happy to PR, but would like to know which of the approaches (or some other) is most palatable.