Skip to content

Commit f8fe301

Browse files
ashwinbashwinb
authored
fix(ci): use test.pypi as extra index for RC dependencies (llamastack#4009)
Backports UV index configuration fixes from `release-0.3.x` (PR llamastack#4002). The main issue: when we created the release branch infrastructure, we configured UV to use `test.pypi` as the PRIMARY index to resolve RC dependencies. This caused UV to look for ALL packages there first, which led to problems - some packages don't have binary wheels on `test.pypi`, so UV tried building from source and failed (like the `psycopg2-binary` issue we hit). The fix is simple: use PyPI as primary (default) and `test.pypi` as an EXTRA index. UV will check PyPI first for everything, and only fall back to `test.pypi` for packages not found there (like our RC client versions). This PR includes: - Fixed `install-llama-stack-client` action to output `UV_EXTRA_INDEX_URL` instead of `UV_INDEX_URL` - New `uv-run-with-index.sh` wrapper that auto-detects release branches and sets UV env vars - Updated pre-commit hooks (`uv-lock`, codegen, etc.) to use the wrapper - Pass UV env vars as Docker build args in all locations - Scope UV env vars properly in Containerfile (inline for llama-stack install, explicitly unset before distribution deps) - Export UV env vars to `GITHUB_ENV` in setup-runner for cross-step persistence The wrapper detects release branches automatically in both CI and local environments, so this "just works" without manual configuration. On main (non-release branch), the wrapper becomes a no-op. Tested and validated on `release-0.3.x` where all CI checks pass.
1 parent 62603d2 commit f8fe301

File tree

10 files changed

+159
-47
lines changed

10 files changed

+159
-47
lines changed

‎.github/actions/install-llama-stack-client/action.yml‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,6 @@ inputs:
88
default: ""
99

1010
outputs:
11-
uv-index-url:
12-
description: 'UV_INDEX_URL to use (set for release branches)'
13-
value: ${{ steps.configure.outputs.uv-index-url }}
1411
uv-extra-index-url:
1512
description: 'UV_EXTRA_INDEX_URL to use (set for release branches)'
1613
value: ${{ steps.configure.outputs.uv-extra-index-url }}
@@ -46,9 +43,8 @@ runs:
4643
exit 1
4744
fi
4845
49-
# Configure to use test.pypi for sync (to resolve RC versions)
50-
echo "uv-index-url=https://test.pypi.org/simple/" >> $GITHUB_OUTPUT
51-
echo "uv-extra-index-url=https://pypi.org/simple/" >> $GITHUB_OUTPUT
46+
# Configure to use test.pypi as extra index (PyPI is primary)
47+
echo "uv-extra-index-url=https://test.pypi.org/simple/" >> $GITHUB_OUTPUT
5248
echo "install-after-sync=true" >> $GITHUB_OUTPUT
5349
echo "install-source=git+https://github.com/llamastack/llama-stack-client-python.git@$BRANCH" >> $GITHUB_OUTPUT
5450
elif [ "${{ inputs.client-version }}" = "latest" ]; then

‎.github/actions/setup-runner/action.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,15 @@ runs:
2727
- name: Install dependencies
2828
shell: bash
2929
env:
30-
UV_INDEX_URL: ${{ steps.client-config.outputs.uv-index-url }}
3130
UV_EXTRA_INDEX_URL: ${{ steps.client-config.outputs.uv-extra-index-url }}
3231
run: |
32+
# Export UV env vars to GITHUB_ENV so they persist across steps
33+
if [ -n "$UV_EXTRA_INDEX_URL" ]; then
34+
echo "UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL" >> $GITHUB_ENV
35+
echo "UV_INDEX_STRATEGY=unsafe-best-match" >> $GITHUB_ENV
36+
echo "Exported UV environment variables for subsequent steps"
37+
fi
38+
3339
echo "Updating project dependencies via uv sync"
3440
uv sync --all-groups
3541

‎.github/workflows/install-script-ci.yml‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,10 +30,16 @@ jobs:
3030

3131
- name: Build a single provider
3232
run: |
33+
BUILD_ARGS="--build-arg INSTALL_MODE=editable --build-arg DISTRO_NAME=starter"
34+
if [ -n "${UV_EXTRA_INDEX_URL:-}" ]; then
35+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL"
36+
fi
37+
if [ -n "${UV_INDEX_STRATEGY:-}" ]; then
38+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_INDEX_STRATEGY=$UV_INDEX_STRATEGY"
39+
fi
3340
docker build . \
3441
-f containers/Containerfile \
35-
--build-arg INSTALL_MODE=editable \
36-
--build-arg DISTRO_NAME=starter \
42+
$BUILD_ARGS \
3743
--tag llama-stack:starter-ci
3844
3945
- name: Run installer end-to-end

‎.github/workflows/pre-commit.yml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,9 +136,12 @@ jobs:
136136

137137
- name: Sync dev + type_checking dependencies
138138
env:
139-
UV_INDEX_URL: ${{ steps.client-config.outputs.uv-index-url }}
140139
UV_EXTRA_INDEX_URL: ${{ steps.client-config.outputs.uv-extra-index-url }}
141140
run: |
141+
if [ -n "$UV_EXTRA_INDEX_URL" ]; then
142+
export UV_INDEX_STRATEGY="unsafe-best-match"
143+
fi
144+
142145
uv sync --group dev --group type_checking
143146
144147
# Install specific client version after sync if needed
@@ -148,7 +151,13 @@ jobs:
148151
fi
149152
150153
- name: Run mypy (full type_checking)
154+
env:
155+
UV_EXTRA_INDEX_URL: ${{ steps.client-config.outputs.uv-extra-index-url }}
151156
run: |
157+
if [ -n "$UV_EXTRA_INDEX_URL" ]; then
158+
export UV_INDEX_STRATEGY="unsafe-best-match"
159+
fi
160+
152161
set +e
153162
uv run --group dev --group type_checking mypy
154163
status=$?

‎.github/workflows/providers-build.yml‎

Lines changed: 28 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -72,10 +72,16 @@ jobs:
7272
- name: Build container image
7373
if: matrix.image-type == 'container'
7474
run: |
75+
BUILD_ARGS="--build-arg INSTALL_MODE=editable --build-arg DISTRO_NAME=${{ matrix.distro }}"
76+
if [ -n "${UV_EXTRA_INDEX_URL:-}" ]; then
77+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL"
78+
fi
79+
if [ -n "${UV_INDEX_STRATEGY:-}" ]; then
80+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_INDEX_STRATEGY=$UV_INDEX_STRATEGY"
81+
fi
7582
docker build . \
7683
-f containers/Containerfile \
77-
--build-arg INSTALL_MODE=editable \
78-
--build-arg DISTRO_NAME=${{ matrix.distro }} \
84+
$BUILD_ARGS \
7985
--tag llama-stack:${{ matrix.distro }}-ci
8086
8187
- name: Print dependencies in the image
@@ -108,12 +114,18 @@ jobs:
108114
- name: Build container image
109115
run: |
110116
BASE_IMAGE=$(yq -r '.distribution_spec.container_image // "python:3.12-slim"' src/llama_stack/distributions/ci-tests/build.yaml)
117+
BUILD_ARGS="--build-arg INSTALL_MODE=editable --build-arg DISTRO_NAME=ci-tests"
118+
BUILD_ARGS="$BUILD_ARGS --build-arg BASE_IMAGE=$BASE_IMAGE"
119+
BUILD_ARGS="$BUILD_ARGS --build-arg RUN_CONFIG_PATH=/workspace/src/llama_stack/distributions/ci-tests/run.yaml"
120+
if [ -n "${UV_EXTRA_INDEX_URL:-}" ]; then
121+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL"
122+
fi
123+
if [ -n "${UV_INDEX_STRATEGY:-}" ]; then
124+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_INDEX_STRATEGY=$UV_INDEX_STRATEGY"
125+
fi
111126
docker build . \
112127
-f containers/Containerfile \
113-
--build-arg INSTALL_MODE=editable \
114-
--build-arg DISTRO_NAME=ci-tests \
115-
--build-arg BASE_IMAGE="$BASE_IMAGE" \
116-
--build-arg RUN_CONFIG_PATH=/workspace/src/llama_stack/distributions/ci-tests/run.yaml \
128+
$BUILD_ARGS \
117129
-t llama-stack:ci-tests
118130
119131
- name: Inspect the container image entrypoint
@@ -148,12 +160,18 @@ jobs:
148160
- name: Build UBI9 container image
149161
run: |
150162
BASE_IMAGE=$(yq -r '.distribution_spec.container_image // "registry.access.redhat.com/ubi9:latest"' src/llama_stack/distributions/ci-tests/build.yaml)
163+
BUILD_ARGS="--build-arg INSTALL_MODE=editable --build-arg DISTRO_NAME=ci-tests"
164+
BUILD_ARGS="$BUILD_ARGS --build-arg BASE_IMAGE=$BASE_IMAGE"
165+
BUILD_ARGS="$BUILD_ARGS --build-arg RUN_CONFIG_PATH=/workspace/src/llama_stack/distributions/ci-tests/run.yaml"
166+
if [ -n "${UV_EXTRA_INDEX_URL:-}" ]; then
167+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL"
168+
fi
169+
if [ -n "${UV_INDEX_STRATEGY:-}" ]; then
170+
BUILD_ARGS="$BUILD_ARGS --build-arg UV_INDEX_STRATEGY=$UV_INDEX_STRATEGY"
171+
fi
151172
docker build . \
152173
-f containers/Containerfile \
153-
--build-arg INSTALL_MODE=editable \
154-
--build-arg DISTRO_NAME=ci-tests \
155-
--build-arg BASE_IMAGE="$BASE_IMAGE" \
156-
--build-arg RUN_CONFIG_PATH=/workspace/src/llama_stack/distributions/ci-tests/run.yaml \
174+
$BUILD_ARGS \
157175
-t llama-stack:ci-tests-ubi9
158176
159177
- name: Inspect UBI9 image

‎.pre-commit-config.yaml‎

Lines changed: 22 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -52,33 +52,20 @@ repos:
5252
additional_dependencies:
5353
- black==24.3.0
5454

55-
- repo: https://github.com/astral-sh/uv-pre-commit
56-
rev: 0.7.20
57-
hooks:
58-
- id: uv-lock
5955

6056
- repo: https://github.com/pre-commit/mirrors-mypy
6157
rev: v1.18.2
6258
hooks:
6359
- id: mypy
6460
additional_dependencies:
6561
- uv==0.6.2
62+
- mypy
6663
- pytest
6764
- rich
6865
- types-requests
6966
- pydantic
70-
- httpx
7167
pass_filenames: false
7268

73-
- repo: local
74-
hooks:
75-
- id: mypy-full
76-
name: mypy (full type_checking)
77-
entry: uv run --group dev --group type_checking mypy
78-
language: system
79-
pass_filenames: false
80-
stages: [manual]
81-
8269
# - repo: https://github.com/tcort/markdown-link-check
8370
# rev: v3.11.2
8471
# hooks:
@@ -87,11 +74,26 @@ repos:
8774

8875
- repo: local
8976
hooks:
77+
- id: uv-lock
78+
name: uv-lock
79+
additional_dependencies:
80+
- uv==0.7.20
81+
entry: ./scripts/uv-run-with-index.sh lock
82+
language: python
83+
pass_filenames: false
84+
require_serial: true
85+
files: ^(pyproject\.toml|uv\.lock)$
86+
- id: mypy-full
87+
name: mypy (full type_checking)
88+
entry: ./scripts/uv-run-with-index.sh run --group dev --group type_checking mypy
89+
language: system
90+
pass_filenames: false
91+
stages: [manual]
9092
- id: distro-codegen
9193
name: Distribution Template Codegen
9294
additional_dependencies:
9395
- uv==0.7.8
94-
entry: uv run --group codegen ./scripts/distro_codegen.py
96+
entry: ./scripts/uv-run-with-index.sh run --group codegen ./scripts/distro_codegen.py
9597
language: python
9698
pass_filenames: false
9799
require_serial: true
@@ -100,7 +102,7 @@ repos:
100102
name: Provider Codegen
101103
additional_dependencies:
102104
- uv==0.7.8
103-
entry: uv run --group codegen ./scripts/provider_codegen.py
105+
entry: ./scripts/uv-run-with-index.sh run --group codegen ./scripts/provider_codegen.py
104106
language: python
105107
pass_filenames: false
106108
require_serial: true
@@ -109,7 +111,7 @@ repos:
109111
name: API Spec Codegen
110112
additional_dependencies:
111113
- uv==0.7.8
112-
entry: sh -c 'uv run ./docs/openapi_generator/run_openapi_generator.sh > /dev/null'
114+
entry: sh -c './scripts/uv-run-with-index.sh run ./docs/openapi_generator/run_openapi_generator.sh > /dev/null'
113115
language: python
114116
pass_filenames: false
115117
require_serial: true
@@ -150,7 +152,7 @@ repos:
150152
name: Generate CI documentation
151153
additional_dependencies:
152154
- uv==0.7.8
153-
entry: uv run ./scripts/gen-ci-docs.py
155+
entry: ./scripts/uv-run-with-index.sh run ./scripts/gen-ci-docs.py
154156
language: python
155157
pass_filenames: false
156158
require_serial: true
@@ -162,6 +164,7 @@ repos:
162164
files: ^src/llama_stack/ui/.*\.(ts|tsx)$
163165
pass_filenames: false
164166
require_serial: true
167+
165168
- id: check-log-usage
166169
name: Ensure 'llama_stack.log' usage for logging
167170
entry: bash
@@ -197,6 +200,7 @@ repos:
197200
echo;
198201
exit 1;
199202
} || true
203+
200204
ci:
201205
autofix_commit_msg: 🎨 [pre-commit.ci] Auto format from pre-commit.com hooks
202206
autoupdate_commit_msg: ⬆ [pre-commit.ci] pre-commit autoupdate

‎containers/Containerfile‎

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@ ARG KEEP_WORKSPACE=""
1919
ARG DISTRO_NAME="starter"
2020
ARG RUN_CONFIG_PATH=""
2121
ARG UV_HTTP_TIMEOUT=500
22+
ARG UV_EXTRA_INDEX_URL=""
23+
ARG UV_INDEX_STRATEGY=""
2224
ENV UV_HTTP_TIMEOUT=${UV_HTTP_TIMEOUT}
2325
ENV PYTHONDONTWRITEBYTECODE=1
2426
ENV PIP_DISABLE_PIP_VERSION_CHECK=1
@@ -45,7 +47,7 @@ RUN set -eux; \
4547
exit 1; \
4648
fi
4749

48-
RUN pip install --no-cache uv
50+
RUN pip install --no-cache-dir uv
4951
ENV UV_SYSTEM_PYTHON=1
5052

5153
ENV INSTALL_MODE=${INSTALL_MODE}
@@ -68,41 +70,50 @@ RUN set -eux; \
6870
echo "LLAMA_STACK_CLIENT_DIR is set but $LLAMA_STACK_CLIENT_DIR does not exist" >&2; \
6971
exit 1; \
7072
fi; \
71-
uv pip install --no-cache -e "$LLAMA_STACK_CLIENT_DIR"; \
73+
uv pip install --no-cache-dir -e "$LLAMA_STACK_CLIENT_DIR"; \
7274
fi;
7375

7476
# Install llama-stack
77+
# Use UV_EXTRA_INDEX_URL inline only for this step to avoid affecting distribution deps
7578
RUN set -eux; \
7679
if [ "$INSTALL_MODE" = "editable" ]; then \
7780
if [ ! -d "$LLAMA_STACK_DIR" ]; then \
7881
echo "INSTALL_MODE=editable requires LLAMA_STACK_DIR to point to a directory inside the build context" >&2; \
7982
exit 1; \
8083
fi; \
81-
uv pip install --no-cache -e "$LLAMA_STACK_DIR"; \
84+
if [ -n "$UV_EXTRA_INDEX_URL" ] && [ -n "$UV_INDEX_STRATEGY" ]; then \
85+
UV_EXTRA_INDEX_URL="$UV_EXTRA_INDEX_URL" UV_INDEX_STRATEGY="$UV_INDEX_STRATEGY" \
86+
uv pip install --no-cache-dir -e "$LLAMA_STACK_DIR"; \
87+
else \
88+
unset UV_EXTRA_INDEX_URL UV_INDEX_STRATEGY; \
89+
uv pip install --no-cache-dir -e "$LLAMA_STACK_DIR"; \
90+
fi; \
8291
elif [ "$INSTALL_MODE" = "test-pypi" ]; then \
83-
uv pip install --no-cache fastapi libcst; \
92+
uv pip install --no-cache-dir fastapi libcst; \
8493
if [ -n "$TEST_PYPI_VERSION" ]; then \
85-
uv pip install --no-cache --extra-index-url https://test.pypi.org/simple/ --index-strategy unsafe-best-match "llama-stack==$TEST_PYPI_VERSION"; \
94+
uv pip install --no-cache-dir --extra-index-url https://test.pypi.org/simple/ --index-strategy unsafe-best-match "llama-stack==$TEST_PYPI_VERSION"; \
8695
else \
87-
uv pip install --no-cache --extra-index-url https://test.pypi.org/simple/ --index-strategy unsafe-best-match llama-stack; \
96+
uv pip install --no-cache-dir --extra-index-url https://test.pypi.org/simple/ --index-strategy unsafe-best-match llama-stack; \
8897
fi; \
8998
else \
9099
if [ -n "$PYPI_VERSION" ]; then \
91-
uv pip install --no-cache "llama-stack==$PYPI_VERSION"; \
100+
uv pip install --no-cache-dir "llama-stack==$PYPI_VERSION"; \
92101
else \
93-
uv pip install --no-cache llama-stack; \
102+
uv pip install --no-cache-dir llama-stack; \
94103
fi; \
95104
fi;
96105

97106
# Install the dependencies for the distribution
107+
# Explicitly unset UV index env vars to ensure we only use PyPI for distribution deps
98108
RUN set -eux; \
109+
unset UV_EXTRA_INDEX_URL UV_INDEX_STRATEGY; \
99110
if [ -z "$DISTRO_NAME" ]; then \
100111
echo "DISTRO_NAME must be provided" >&2; \
101112
exit 1; \
102113
fi; \
103114
deps="$(llama stack list-deps "$DISTRO_NAME")"; \
104115
if [ -n "$deps" ]; then \
105-
printf '%s\n' "$deps" | xargs -L1 uv pip install --no-cache; \
116+
printf '%s\n' "$deps" | xargs -L1 uv pip install --no-cache-dir; \
106117
fi
107118

108119
# Cleanup

‎scripts/docker.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -215,6 +215,16 @@ build_image() {
215215
--build-arg "LLAMA_STACK_DIR=/workspace"
216216
)
217217

218+
# Pass UV index configuration for release branches
219+
if [[ -n "${UV_EXTRA_INDEX_URL:-}" ]]; then
220+
echo "Adding UV_EXTRA_INDEX_URL to docker build: $UV_EXTRA_INDEX_URL"
221+
build_cmd+=(--build-arg "UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL")
222+
fi
223+
if [[ -n "${UV_INDEX_STRATEGY:-}" ]]; then
224+
echo "Adding UV_INDEX_STRATEGY to docker build: $UV_INDEX_STRATEGY"
225+
build_cmd+=(--build-arg "UV_INDEX_STRATEGY=$UV_INDEX_STRATEGY")
226+
fi
227+
218228
if ! "${build_cmd[@]}"; then
219229
echo "❌ Failed to build Docker image"
220230
exit 1

‎scripts/integration-tests.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,16 @@ if [[ "$STACK_CONFIG" == *"docker:"* && "$COLLECT_ONLY" == false ]]; then
279279
--build-arg "LLAMA_STACK_DIR=/workspace"
280280
)
281281

282+
# Pass UV index configuration for release branches
283+
if [[ -n "${UV_EXTRA_INDEX_URL:-}" ]]; then
284+
echo "Adding UV_EXTRA_INDEX_URL to docker build: $UV_EXTRA_INDEX_URL"
285+
build_cmd+=(--build-arg "UV_EXTRA_INDEX_URL=$UV_EXTRA_INDEX_URL")
286+
fi
287+
if [[ -n "${UV_INDEX_STRATEGY:-}" ]]; then
288+
echo "Adding UV_INDEX_STRATEGY to docker build: $UV_INDEX_STRATEGY"
289+
build_cmd+=(--build-arg "UV_INDEX_STRATEGY=$UV_INDEX_STRATEGY")
290+
fi
291+
282292
if ! "${build_cmd[@]}"; then
283293
echo "❌ Failed to build Docker image"
284294
exit 1

0 commit comments

Comments
 (0)