[pull] main from astral-sh:main#55
Merged
Merged
Conversation
Closes #1495. Partially addresses #7642. ## Summary Adds centralized virtual environment storage behind the `centralized-project-envs` preview feature. When enabled, default project virtual environments (normally stored in `.venv`) are stored in the `environments-v2` cache bucket instead of locally in the project directory. A `.venv` symlink/junction is maintained as a compatibility pointer for editor/IDE/tool discovery and activation. There is no support for `uv venv` or a `.venv` path file fallback in this version. These are deferred for follow-up work to keep this PR easier to review. When `--no-cache` is enabled, uv warns and uses the ordinary local `.venv` behavior for that invocation. The feature is intended to improve performance when switching environments and when working on a project directory which is on a different filesystem to the cache. Keeping separate cached environments for each interpreter allows reuse and fast switching. Moving the environment into the cache guarantees environments will be on the same filesystem as package files which should almost guarantee hardlinks can be used. There are no current plans to support standalone environments not tied to a specific project. Note: Most of the added lines in this PR are tests. ## Design ### Activation Criteria When the feature is enabled, a centralized environment is used only if `UV_PROJECT_ENVIRONMENT` is not set and an active environment is not selected with `--active`. The feature has no effect when `--no-cache` is enabled, and uv falls back to ordinary local environment behavior. ### Environment Reuse The cache key used to locate the centralized environment is derived from the workspace path and interpreter identity. Upgradeable uv-managed Python installations use a minor-version identity so their environments can be reused after a transparent patch upgrade. For other environments, the identity is the interpreter key (which contains the implementation, exact version, operating system, architecture, libc, and variant). The result is a partially human-readable cache key in the form: `<name>-<implementation><python-version>-<hash>`. The implementation portion uses abbreviations such as `cp` for CPython, `pp` for PyPy, and `gp` for GraalPy, and currently `pyodide` for Pyodide. If the project name is not available, the directory name is used; if neither is available, the name component is omitted (resulting in `<implementation><python-version>-<hash>`). The project or directory name component is slugified and limited to 100 characters to avoid hitting filesystem path-component length limits. The Python version component normally includes the major, minor, and patch versions, along with any pre-release identifier. For upgradeable uv-managed Python installations, the patch version is omitted. Examples: * `my-project-cp3.12.4-0123456798abcdef` * `my-project-cp3.12-0123456789abcdef` for an upgradeable managed Python * `cp3.14.0b2-fedcba8976543210` ### Interpreter Discovery The Python interpreter must be known to calculate the cache key. It can be discovered based on the current Python request but this is a comparatively slow process and would be a big performance hit on consecutive `uv run` invocations. The compatibility `.venv` is (currently) inspected as a potential environment. This environment's interpreter is then used to calculate the correct cache key. This two-step lookup avoids a broken situation where a user-provided link points outside of the cache, or at a cache entry for the wrong project. ### The `.venv` Link To aid in ensuring compatibility with existing third-party tools (and ty), an attempt is made to create a symlink or junction from `.venv` to the centralized environment location. This can fail in some system configurations. Failure is non-fatal because uv can continue using the centralized environment directly. This leads to user-facing warnings when encountered by `uv sync`, `uv add`, `uv remove`, `uv version`, `uv check`, and `uv workspace metadata --sync`. To avoid warning on every invocation, `uv run` logs publication failures with `warn!` instead. A later PR will make symlink/junction creation failure fall back to a plain file containing the environment path. This will preserve uv's interpreter discovery fast path for these cases. If/when support for such files lands in third-party projects, they will also be able to use this for environment discovery. A real local virtual environment at `.venv` is removed without prompting. An existing symlink or junction is replaced without following it. An unrelated non-empty directory is not deleted; link creation fails with a warning instead. On Windows, empty directories may be replaced to clean up copied junction remnants. During non-dry-run project operations, the link is eagerly updated whenever a centralized environment is created or reused. A later PR may limit this replacement to cases where it's actually necessary. ### Interoperability / Safety In order to allow uv to function seamlessly _after_ the feature is deactivated on a project, changes have been made to the baseline behaviour of uv: if a symlink or junction is found to point into the cache and uv would normally clear it, the symlink/junction is deleted, not the target. Previously, uv would follow this symlink and re-create the target in place. If the link points at a compatible environment, it will continue to be used as normal. A non-empty `.venv` directory that does not appear to be a virtual environment is left untouched. ### Cache Lifecycle Both `uv cache clean` and `uv cache prune` will delete all centralized environments unconditionally. This leaves `.venv` links dangling intentionally. This is recoverable and will lead to re-creation without warnings or errors. ## Test Plan A combination of existing test coverage and a bunch of new tests. A separate test suite was used to gain more confidence in thie PR. This test suite was used to inform the selection of tests added in this PR. An attempt was made not to duplicate coverage within the added tests. ### VS Code An extensive suite of automated VS Code tests was created and ran on MacOS and Windows (Linux coming eventually although static analysis showed no reason for any deviation and past experience confirms this). The following is a representative snapshot of the current state: * VS Code finds centralized environments correctly using the symlink/junction. * Normal `uv add` and `uv remove` workflows work with Pylance and ty. * Pylance can miss package changes made directly to a centralized environment (`uv pip`); ty handles them correctly. * Replacing an environment can leave both tools stale, but this already happens with regular `.venv` directories. * Overall, centralized environments work for normal use, with a small number of existing or fixable editor issues. ### Outstanding issues/questions Should the venv be keyed further on choice of extras/groups, the project inside a workspace? It would increase the number of environments, but also allow environment reuse without package selection changes in more cases. When upgrading from a local environment, we make no attempt to preserve the chosen python interpreter. Mainly because we don't have an easy route to find its provenance so we can re-use it after deletion. We could use the version of the interpreter as a lookup hint, though. Should we?
## Summary <img width="186" height="176" alt="image" src="https://github.com/user-attachments/assets/37ba708f-8a1d-44db-b7fa-c03984c31eda" /> I noticed this while working on #19991. I _believe_ we don't need a version range on `uv_build`, since (IIUC) it'll select uv's own built-in version. ## Test Plan Updated the snapshots. --------- Signed-off-by: William Woodruff <william@yossarian.net>
## Summary
`override-dependencies` currently applies every override globally, which
makes it impossible to patch a dependency requirement for one dependent
package version without also changing direct requirements and
requirements from other packages.
This allows structured entries alongside existing requirement strings:
```toml
override-dependencies = [
"idna==3.1",
{ package = { name = "anyio", version = "3.7.0" }, dependencies = ["idna==3.2"] },
]
```
A structured entry replaces requirements for the listed dependencies
only when resolving dependency metadata for the matching package. The
version is optional, matching all versions when omitted; an
exact-version entry takes precedence over a versionless entry, and
scoped requirements take precedence over global overrides for the same
dependency.
Scoped overrides currently support registry version specifiers only.
Direct URL, path, and Git sources and explicit indexes are rejected with
targeted errors rather than being applied without matching package
context.
Pre-release and yanked-version permissions are initialized before
resolution determines which package scopes apply, so an explicit
pre-release or yanked-version pin in any scoped override opts that
dependency into the corresponding candidate-selection behavior for the
entire resolution, even if the scope is not selected.
## Summary
Package-scoped overrides currently only replace dependencies that the
selected parent already declares. A scoped requirement for a new
dependency is otherwise silently ignored.
This treats scoped `dependencies` entries as partial metadata patches:
requirements with matching names replace existing declarations, while
requirements with new names are appended to the selected package's
dependencies. Global overrides remain non-additive, and dependencies
omitted from the scoped entry remain unchanged.
For example:
```toml
[tool.uv]
override-dependencies = [
{ package = { name = "anyio", version = "3.7.0" }, dependencies = ["idna==3.2", "typing-extensions==4.10.0"] },
]
```
For `anyio==3.7.0`, this replaces the existing IDNA requirement and adds
a new dependency on `typing-extensions`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )