Skip to content

Add a --clean-cache command to clean up locations specified at CIBW_CACHE_PATH #2489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

agriyakhetarpal
Copy link
Member

As described in #2397, this PR adds a --clean-cache command to clean CIBW_CACHE_PATH. It does not implement granular cache cleans at the moment (it does not clean per supported platform), and shutil.rmtree()s the entire folder. I added three tests that account for possible code paths: if cache files exist, if they don't, or if the cache files are somehow in a non-readable/writable location (though I think that aspect should also be validated at the creation time of the cache paths).

This is especially useful for Pyodide builds, as the Emscripten SDK can occupy up to 1.5 GiB of space; however, it is also beneficial for builds on all other platforms.

Closes #2397

@@ -421,6 +424,71 @@ def test_debug_traceback(monkeypatch, method, capfd):
assert "Traceback (most recent call last)" in err


def test_clean_cache_when_cache_exists(tmp_path, monkeypatch, capfd):
monkeypatch.undo()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to add monkeypatch.undo() here and in the tests below because fake_package_dir_autouse is autoused and applied to all tests, and it breaks these tests because I'm monkeypatching CIBW_CACHE_PATH.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. could we move these tests to a new module main_commands_test.py (--print_build_identifiers tests can come too, if they exist), and then scope the autouse fixture to only apply to tests in main_options_test.py, main_platform_test.py, and main_requires_python_test.py?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@agriyakhetarpal Did you want to do this? Or should I add it to my list of things I'll do if there's a lull in the SciPy sprints? :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder, @henryiii – I am onboard with @joerick's idea here. I think this is indeed something a new contributor/you can take at the sprints, so I am okay with it :D

Copy link
Contributor

@joerick joerick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together!

@@ -421,6 +424,71 @@ def test_debug_traceback(monkeypatch, method, capfd):
assert "Traceback (most recent call last)" in err


def test_clean_cache_when_cache_exists(tmp_path, monkeypatch, capfd):
monkeypatch.undo()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. could we move these tests to a new module main_commands_test.py (--print_build_identifiers tests can come too, if they exist), and then scope the autouse fixture to only apply to tests in main_options_test.py, main_platform_test.py, and main_requires_python_test.py?

Signed-off-by: Henry Schreiner <[email protected]>
@henryiii
Copy link
Contributor

The one thought I had was could we check for something specific to make sure this is in fact a cibuildwheel cache dir? Just to protect for a mistake like setting the cache dir to something like $HOME. :)

@agriyakhetarpal
Copy link
Member Author

The one thought I had was could we check for something specific to make sure this is in fact a cibuildwheel cache dir? Just to protect for a mistake like setting the cache dir to something like $HOME. :)

Do you mean we should check if CIBW_CACHE_PATH contains cibuildwheel or if it matches platformdirs.user_cache_path()?

Realistically, I think we'd have no defense against this unless we make the cache path non-configurable and drop the ability to set it via the DEFAULT_CIBW_CACHE_PATH environment variable. This is unless I've misunderstood you here :)

@henryiii
Copy link
Contributor

I was thinking looking for a specific file or folder that would exist if it was cibuildwheel's cache.

@agriyakhetarpal
Copy link
Member Author

Right, we could include a sentinel .cibuildwheel_cached file inside the cache folder; we do similar things with Pyodide's xbuildenv cache.

@agriyakhetarpal agriyakhetarpal marked this pull request as draft July 18, 2025 23:39
@joerick
Copy link
Contributor

joerick commented Jul 19, 2025

Right, we could include a sentinel .cibuildwheel_cache file inside the cache folder; we do similar things with Pyodide's xbuildenv cache.

That's a nice idea. Might save somebody's day.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cache clean command
3 participants