perf: reduce import time by deferring heavy dependencies

[leotrs]

Summary

• Defer scipy.stats, scipy.special, scipy.sparse.linalg, pandas, matplotlib.colors, requests, and networkx from module-level to function-level imports where they are only used in specific methods
• Restructure to_hypergraph/to_simplicial_complex in convert/higher_order_network.py so common input types (list, dict, Hypergraph) are handled before falling through to a helper that imports pandas/scipy for DataFrame/matrix inputs
• Add regression test asserting import xgi completes in under 3 seconds

Addresses #651.

Files changed

• xgi/stats/__init__.py — defer scipy.stats.moment, pandas
• xgi/utils/utilities.py — defer pandas, requests, matplotlib.colors
• xgi/algorithms/properties.py — defer scipy.special.comb
• xgi/algorithms/simpliciality.py — defer scipy.special.binom
• xgi/algorithms/centrality.py — defer scipy.sparse.linalg.eigsh, networkx
• xgi/convert/graph.py, line_graph.py, bipartite_graph.py, encapsulation_dag.py — defer networkx
• xgi/convert/higher_order_network.py — defer pandas, scipy.sparse, numpy.matrix
• xgi/convert/pandas.py — defer pandas
• xgi/generators/simplicial_complexes.py — defer networkx, scipy.special.comb
• tests/test_import_time.py — new regression test

Test plan

• All 392 tests pass + 1 new import time regression test
• Measured import time reduction from ~3s to ~1.5s on macOS/arm64

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 98.11321% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.68%. Comparing base (6e7fe50) to head (ebb6929).
⚠️ Report is 19 commits behind head on dev.

Files with missing lines	Patch %	Lines
xgi/stats/__init__.py	83.33%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #692      +/-   ##
==========================================
+ Coverage   93.59%   93.68%   +0.09%     
==========================================
  Files          66       66              
  Lines        5120     5133      +13     
==========================================
+ Hits         4792     4809      +17     
+ Misses        328      324       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kaiser-dan]

Thanks @leotrs , this should speed things up nicely!

I left a few comments, mostly stylistic, but one in higher_order_network.py concerns error propagation and is more design-oriented.

[kaiser-dan]

I believe these should remain outside of the public API.

[leotrs]

These are user-facing I/O functions (xgi.read_json() / xgi.write_json()) — they should be part of the public API. The __all__ in readwrite/__init__.py feeds into xgi/__init__.py's __all__, which is how they end up as top-level exports. They were just missing from the list.

[kaiser-dan]

Didn't we decide to remove these from the public API in this thread?

[kaiser-dan]

I'm not crazy about the else branch passing to an internal helper which then handles any errors. If we extended the elif branches to check for the remaining supported types then dispatch the correct helper, we could keep all of the dispatch and supported type logic in the one function which may make deciphering stack traces clearer. Specifically, any stack trace originating from to_hypergraph is from the point of conversion dispatch, e.g. an unsupported data type, and any errors from the internal helpers is from errors within a supported type.

[leotrs]

Fair point about stack traces. The tradeoff here is that inlining the `isinstance` checks back into `to_hypergraph` would require importing `pd.DataFrame` and all the scipy sparse types at function entry, which defeats the lazy-import purpose. The helper adds one extra frame to the stack trace, but keeps the heavy imports out of the common path. I think the performance win justifies the indirection, but open to other ideas if you see a cleaner way to structure it.

[kaiser-dan]

Hmm, no cleaner solution comes to mind. Related to recent #691 and the discussion therein, I am wondering if we would benefit from a more verbose exception hierarchy? The connection here is maybe sort of XGIDataError for data related issues for higher-order interactions in particular (e.g. this unsupported data type for conversions, dihyperedges with empty heads/tails, maybe others?).

Without extending the exception hierarchy, I can't think of "nice" ways to maintain error legibility, so I say we keep the performance improvement and move on.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB