[pull] main from astral-sh:main#57
Merged
Merged
Conversation
Adds a self-contained workspace-discovery benchmark with 127 members and large generated TOML files. This benchmark targets the internal workspace discovery functions and caching, not a real `uv run` case.
`ResolverOutput` consumers independently traverse the graph to find base distributions. This PR adds a crate-private `base_dists()` iterator and uses it for lock construction, pylock export, `len()`, and `is_empty()` without changing graph semantics. Later proxy-index work uses the same `(NodeIndex, &AnnotatedDist)` view to canonicalize lock artifacts. This is a pure refactor with no user-visible changes. Depends on #20001.
…19933) When using `tool.uv.environments`, uv would repeat the combined marker of the `tool.uv.environments` universe for all dependency edges. We can remove this redundant marker, improving lockfile readability: **Before** ```toml [[package]] name = "accelerate" version = "0.33.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "huggingface-hub", marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, { name = "numpy", marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, { name = "packaging", marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, { name = "psutil", marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, { name = "pyyaml", marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, { name = "safetensors", marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin') or (platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin'" }, { name = "torch", version = "2.4.0", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/d4/24/5e813a41495ec7fdbc6a0f08e38c099caccf49147b8cd84053f4c3007c35/accelerate-0.33.0.tar.gz", hash = "sha256:11ba481ed6ea09191775df55ce464aeeba67a024bd0261a44b77b30fb439e26a", size = 314567, upload-time = "2024-07-23T13:13:57.691Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/15/33/b6b4ad5efa8b9f4275d4ed17ff8a44c97276171341ba565fdffb0e3dc5e8/accelerate-0.33.0-py3-none-any.whl", hash = "sha256:0a7f33d60ba09afabd028d4f0856dd19c5a734b7a596d637d9dd6e3d0eadbaf3", size = 315131, upload-time = "2024-07-23T13:13:55.502Z" }, ] ``` **After** ```toml [[package]] name = "accelerate" version = "0.33.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "huggingface-hub" }, { name = "numpy" }, { name = "packaging" }, { name = "psutil" }, { name = "pyyaml" }, { name = "safetensors" }, { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'darwin'" }, { name = "torch", version = "2.4.0", source = { registry = "https://pypi.org/simple" }, marker = "(platform_machine == 'x86_64' and platform_python_implementation == 'CPython' and sys_platform == 'linux') or (platform_machine == 'AMD64' and platform_python_implementation == 'CPython' and sys_platform == 'win32')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/d4/24/5e813a41495ec7fdbc6a0f08e38c099caccf49147b8cd84053f4c3007c35/accelerate-0.33.0.tar.gz", hash = "sha256:11ba481ed6ea09191775df55ce464aeeba67a024bd0261a44b77b30fb439e26a", size = 314567, upload-time = "2024-07-23T13:13:57.691Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/15/33/b6b4ad5efa8b9f4275d4ed17ff8a44c97276171341ba565fdffb0e3dc5e8/accelerate-0.33.0-py3-none-any.whl", hash = "sha256:0a7f33d60ba09afabd028d4f0856dd19c5a734b7a596d637d9dd6e3d0eadbaf3", size = 315131, upload-time = "2024-07-23T13:13:55.502Z" }, ] ``` Resolving the transformers test case with the settings shown below, the lockfile shrinks from 920KB to 716KB, a 22% reduction. ```toml [tool.uv] exclude-dependencies = ["decord"] environments = [ "(platform_python_implementation == 'CPython' and platform_machine == 'x86_64' and sys_platform == 'linux')", "(platform_python_implementation == 'CPython' and platform_machine == 'AMD64' and sys_platform == 'win32')", "(platform_python_implementation == 'CPython' and platform_machine == 'x86_64' and sys_platform == 'darwin')", ] ``` This change causes the lockfile to churn significantly for all `tool.uv.environments` users.
## Summary #19933 omits dependency markers that are true throughout the configured environment, but a conditional dependency still repeats that environment in full. For example, an edge represented internally as `<environment> and python_version < '3.11'` writes the complete environment marker instead of only the Python-version condition. This adds general marker simplification under an assumption to the marker algebra. Lock serialization simplifies each dependency marker under the resolution environment, while deserialization conjoins that environment again to restore the marker's standalone meaning. This lets us persist the residual condition without changing behavior inside the supported environment.
`uv check --script` currently ignores a direct `ty` dependency from PEP 723 metadata when choosing the checker. The script environment contains the locked package, but uv still selects a standalone `ty` from its built-in compatible range, so the checker can differ from the script’s dependency graph. This extends `Lock::dependency_selection` to include applicable lock-manifest requirements. When a script directly declares `ty`, `uv check` now runs the marker-selected locked package from the script environment, preserving its source and transitive dependencies without another resolution. This is a followup to #19884 and it depends on #19982.
The installer planner currently inspects raw `RegistryWheelIndex` entries to decide whether a cached registry distribution satisfies a resolved requirement. This moves those checks into `RegistryWheelIndex::wheel` and `RegistryWheelIndex::source`, then makes `IndexEntry` and the raw `get` iterator private. Proxy indexes separate a package’s canonical index identity from the physical endpoint used to serve its artifacts. This refactor makes `RegistryWheelIndex` the boundary for selecting cached artifacts, allowing subsequent proxy-index changes to search and validate canonical and physical cache locations without adding proxy-specific logic to the installer planner. This is a pure refactor, no functionality change. Depends on #19996
This is the first preparatory PR in the proxy-index stack. Later changes need `RegistryClient` to retain the complete index configuration, but it currently clones `IndexLocations` into a narrower `IndexUrls` representation and splits index state across two types. This PR removes `IndexUrls` and makes `IndexLocations` the single source of truth. `uv add` now uses `IndexLocations::defined_indexes()` directly when persisting command-line indexes. `RegistryClientBuilder::build` and `wrap_existing` now share one construction path, so both cache credentials, apply index middleware, and retain the same locations. This is a pure refactor, there should be no user-visible behavior changes.
Signed-off-by: William Woodruff <william@yossarian.net>
The locked `uv check` tool path materializes the `ty` package selected from `uv.lock` into a cached environment. Hashes from the lockfile contribute to the cache key, but this path did not verify them when downloading and installing the package. This PR fixes it.
Signed-off-by: William Woodruff <william@yossarian.net>
## Summary With the centralised project environments preview feature enabled, `uv venv` invoked with no path and `UV_PROJECT_ENVIRONMENT` unset will now use the environment cache bucket as the backing store for the environment. The key selection logic is the same as for centralised project environments. There's are some cases of note: * With the feature enabled: * Unless you explicitly pass `--no-clear`, existing environments will be clobbered without prompting * If you pass `--allow-existing` this will avoid clearing the _centralised_ environment, before re-using it, but will still clobber local environments when creating the link. This might sound weird, but the alternatives are: * Disable the feature if `.venv` exists, but if `.venv` doesn't exist, instead operate on any "existing" environment in the cache (this gets more confusing the more you think about it) * Don't touch `.venv` if it exists, but otherwise operate on any "existing" environment in the cache * Complain if `.venv` exists but is not referencing the cache (somewhat sane sounding... except also somewhat heavy and still weird) * With the feature disabled: * Whenever we would normally clear an environment, if it's a link to the cache, we just delete the link (we would normally clear the target of the link). I consider this not to be breaking because a `.venv` that is a link into the cache should never have been a thing prior to the feature. * The user is not prompted to remove links to the cache. See also #18214. ## Test Plan A range of new tests covering a variety of normal and corner cases.
## Summary - document the boolean result of `update_project_environment_link` - clarify that the result reports whether `.venv` was updated successfully - follow up on [review feedback from #19912](#19912 (comment)) This is an internal documentation-only change with no runtime behavior impact. ## Test plan - `cargo fmt --all -- --check` - `git diff --check` Co-authored-by: Zanie Blue <contact@zanie.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )