-
Notifications
You must be signed in to change notification settings - Fork 80
Closed
Labels
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is needed
Description
🐛 Bug
Title: zstd.ZstdError missing for Python < 3.14 fallback import
Description:
In src/litdata/raw/indexer.py, LitData uses the standard library compression.zstd module for Python 3.14+, which correctly provides ZstdError.
For Python versions < 3.14, the code falls back to:
import zstdHowever, the PyPI zstd package does not define ZstdError — it only provides zstd.Error.
This makes the current exception handling inconsistent and can lead to an AttributeError when referencing zstd.ZstdError.
Expected behavior
The fallback zstd implementation for Python < 3.14 should expose ZstdError or the code should adjust to handle the correct exception type for the chosen package.
Actual behavior
AttributeError: module 'zstd' has no attribute 'ZstdError'
(when running Python < 3.14)
Related discussion
Discussion: #765
Environment
- Python version: < 3.14
- litdata version: latest main
- OS: Any
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workinghelp wantedExtra attention is neededExtra attention is needed