feat: add support to disable external version checks

[sanggusti]

Fixes #734

Add mechanism to disable external version checks to address #734

• Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

What does this PR do?

Fixes #734 . Implemented optional environment variable LITDATA_DISABLE_VERSION_CHECK=1 that, when set, skips the PyPI API calls entirely. This prevents delays, timeouts, and unexpected network activity in restricted environments while maintaining backward compatibility (i.e., the default behavior remains unchanged unless the flag is explicitly enabled).

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[sanggusti]

The tests is made by copilot. Seems overtesting but I can delete unnecessary checks if requested.

[tchaton]

Hey @sanggusti Thanks for your first contributions. I am curious to hear what you are using LitData for.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80%. Comparing base (359bbf1) to head (50a9c46).

Additional details and impacted files

@@         Coverage Diff         @@
##           main   #737   +/-   ##
===================================
  Coverage    80%    80%           
===================================
  Files        52     52           
  Lines      7330   7332    +2     
===================================
+ Hits       5869   5874    +5     
+ Misses     1461   1458    -3     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bhimrazy]

Thanks for the pr, @sanggusti! I’ve added a few suggestions.
Please feel free to reach out if you have any questions.

[sanggusti]

Hi @tchaton !

I was looking for a tool for curating/transforming/streaming data for my medical llm finetuning project, and it happens that I'm a fan of lightning. I've just used litdata for the past week and even though I can't use lightning cloud (Indonesia(+62) is not supported yet) I happen to find some issue that I think I can contribute.

[Copilot]

Pull Request Overview

This PR implements a mechanism to disable external version checks to address issue #734, allowing users to skip PyPI API calls in restricted environments.

• Adds environment variable LITDATA_DISABLE_VERSION_CHECK to control version checking behavior
• Updates the version check logic to respect the new flag and skip network requests when disabled
• Includes comprehensive test coverage for the new functionality

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
src/litdata/constants.py	Adds new environment variable constant for disabling version checks
src/litdata/helpers.py	Modifies version check logic to respect the disable flag
tests/test_helper.py	Adds comprehensive tests for the new version check behavior
src/litdata/CHANGELOG.md	Documents the new feature in the changelog

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[deependujha]

blocked by #738

[sanggusti]

Wow, seems like my test format is pretty bad that @deependujha had to overhaul it. Gotta learn more with monkeypatch and patches it seems. May I know why the unrelated tests is failing on CI?

[deependujha]

Hi @sanggusti ,

Thanks so much for contributing to LitData and helping make the project even more awesome! 🙌

A few notes on the tests:

• instead of having separate tests for each condition, parameterizing them can make things more compact and readable.
• Also, it’s better not to read or write environment variables directly with os in tests. Using monkeypatch avoids flaky changes that can interfere with other tests and save you headaches tracking down failures.

The CI failures you’re seeing are due to the new release of torchaudio; PR #738 is expected to fix this.

Really appreciate you taking the time to contribute!

[sanggusti]

The more that I have to learn. Thank you for reviewing and showing how it had to be done @deependujha @bhimrazy 🙌🏻