KatieB5/switch to uv for updating dependencies#3065
Open
KatieB5 wants to merge 7 commits into
Open
Conversation
KatieB5
force-pushed
the
KatieB5/switch-to-uv-for-updating-dependencies
branch
4 times, most recently
from
9486781 to
56ef83c
Compare
4 tasks
This commit adds a uv version and opencodelists project metadata to the pyproject.toml file (https://docs.astral.sh/uv/concepts/projects/config/#configuring-projects). The content of the metadata was aligned to what had been included for jobserver (https://github.com/opensafely-core/job-server/blob/main/pyproject.toml).
1. Import production dependencies with `uv add -r requirements.prod.in -c requirements.prod.txt` Using `-c` to preserve previous dependency versions. This step updated `pyproject.toml` and generated a new `uv.lock` file. 2. Import development dependencies with `sed '/^-r /d' requirements.dev.in | uv add --dev -r - -c requirements.dev.txt`. This means we don't re-declare prod dependencies in the dev group in our `pyproject.toml`. Constraints again ensured all dev versions matched the previous versions in the `requirements.dev.txt` file. 3. Remove `requests` from `dev` group in the `[dependency-groups]` table of pyproject.toml. In a later commit, our just recipes will be updated such that dependancies will be installed into the environment by `uv sync` which by default includes the prod dependencies listed in the `[dependencies]` table and the `dev` dependency group from the `[dependency-groups]` table. https://docs.astral.sh/uv/concepts/projects/dependencies/#default-groups 4. Copy across any dependancy-specific comments from the `.in` files
Update justfile recipes to use the `pyproject.toml` + `uv.lock` dependency workflow via `uv sync` and `uv lock` commands. This broadly matches the approach in airlock (https://github.com/opensafely-core/airlock/blob/main/justfile), repo-template (https://github.com/opensafely-core/repo-template/blob/main/justfile), and jobserver (https://github.com/opensafely-core/job-server/blob/main/justfile). - As part of this work, legacy recipes and requirements were also removed (`_compile`, `requirements-prod`, `requirements-dev`); the exported `$PIP` env var; references to `requirements*.in` / `requirements*.txt`. - Updated `uv.lock` files produced during testing were not committed to avoid unintended dependency updates outside of the team's usual workflow. Testing done: - `just clean`, `just prodenv` - `just clean`, `just devenv`, `just run` - `just upgrade-package django` - `just update-dependencies` - `just test` All commands behaved as expected.
`Dockerfile` updates: - Based on airlock (https://github.com/opensafely-core/airlock/blob/main/docker/Dockerfile) and jobserver (https://github.com/opensafely-core/job-server/blob/main/docker/Dockerfile) `.dockerignore` update: - Removed `pyproject.toml` as we are now making use of this file for dependency management with uv A .keep file was removed when testing the updated `just docker-` recipes. Tested recipes: - Just docker-build & docker-build prod - Just docker-test - Just docker-serve (manually tested the site)
Following switch to uv for dev and prod dependency management, we no longer need these files.
1. Update GHA workflows to configure the OpenSAFELY setup-action with `uv`-based dependency management Previously the `setup-action` defaulted to `pip` caching, which doesn't reflect opencodelist's dependency management approach now that we've switched to uv for both local dev and in our Docker set-up. Most workflow steps execute commands inside Docker containers, so this change is largely for consistency and clarity rather than functionality. Explicitly installing `uv` and enabling `uv` caching makes the workflows declarative about the tooling opencodelists uses, and aligns them with our dependency management approach elsewhere in opencodelists, and other services e.g. jobserver. 2. Remove the `- python-version: "3.12"` config from the dependency update workflow file. The GHA runners currently use Ubuntu 24.04 (`ubuntu-latest`), which provides Python 3.12.3 as the default system Python. Since the workflow now installs `uv` directly, `uv` can discover and use this compatible system Python automatically.
Now that we've switched to `uv` for dependency management throughout our opencodelists infrastructure, we no longer need `pip-tools`.
KatieB5
force-pushed
the
KatieB5/switch-to-uv-for-updating-dependencies
branch
from
56ef83c to
cb93fc3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses part of #2456 . The rest of the work (outlined below) will be addressed in subsequent PRs.
Work done in this PR
pyproject.tomland generateuv.lockfilejustfilerecipesuvpip-toolsdependencyWork to be done in subsequent PR(#3075):
pyproject.tomlfor dependencies we don't explicitly need to constrain after implementing dependency cooldown