Skip to content

test(e2e): migrate upgrade stale sandbox to vitest #5542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cv merged 12 commits into from
Jun 20, 2026
+759 −3
Merged

test(e2e): migrate upgrade stale sandbox to vitest #5542

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
8aefe40
test(e2e): migrate upgrade stale sandbox to vitest
cv Jun 18, 2026
96ad75a
Merge branch 'main' into e2e-migrate/test-upgrade-stale-sandbox
cv Jun 19, 2026
39cf882
Merge branch 'main' into e2e-migrate/test-upgrade-stale-sandbox
cv Jun 19, 2026
7256827
Merge remote-tracking branch 'origin/main' into e2e-migrate/test-upgr…
cv Jun 19, 2026
0f8ccbd
test(e2e): refresh upgrade stale sandbox workflow
cv Jun 19, 2026
bce7dac
Merge remote-tracking branch 'origin/e2e-migrate/test-upgrade-stale-s…
cv Jun 19, 2026
2354010
test(e2e): relax workflow inventory timeout
cv Jun 19, 2026
553645c
test(e2e): address stale upgrade review feedback
cv Jun 19, 2026
7cb5562
test(e2e): move scenario logic out of test wrapper
cv Jun 20, 2026
31a00b5
Merge remote-tracking branch 'origin/e2e-migrate/test-upgrade-stale-s…
cv Jun 20, 2026
97d8faa
test(e2e): restore scenario logic to test file
cv Jun 20, 2026
70cfd1e
test(e2e): move upgrade stale setup branches to helpers
cv Jun 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .github/workflows/e2e-vitest-scenarios.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1695,6 +1695,108 @@ jobs:
if-no-files-found: ignore
retention-days: 14

upgrade-stale-sandbox-vitest:
needs: generate-matrix
if: ${{ (inputs.jobs == '' && inputs.scenarios == '') || contains(format(',{0},', inputs.jobs), ',upgrade-stale-sandbox-vitest,') || contains(format(',{0},', inputs.scenarios), ',upgrade-stale-sandbox,') }}
runs-on: ubuntu-latest
timeout-minutes: 55
env:
FREE_STANDING_VITEST_JOB: "1"
FREE_STANDING_SCENARIO_ID: "upgrade-stale-sandbox"
E2E_ARTIFACT_DIR: ${{ github.workspace }}/e2e-artifacts/vitest/upgrade-stale-sandbox
NEMOCLAW_CLI_BIN: ${{ github.workspace }}/bin/nemoclaw.js
NEMOCLAW_RUN_E2E_SCENARIOS: "1"
NEMOCLAW_NON_INTERACTIVE: "1"
NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE: "1"
NEMOCLAW_SANDBOX_NAME: "e2e-upgrade-stale"
OPENSHELL_GATEWAY: "nemoclaw"
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false

- name: Configure isolated Docker auth directory
run: echo "DOCKER_CONFIG=${RUNNER_TEMP}/docker-config-upgrade-stale-sandbox" >> "$GITHUB_ENV"

- name: Authenticate to Docker Hub
env:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
shell: bash
run: |
set -euo pipefail
if [[ -z "${DOCKERHUB_USERNAME}" || -z "${DOCKERHUB_TOKEN}" ]]; then
echo "::notice::Docker Hub credentials not configured; continuing with anonymous pulls."
exit 0
fi
mkdir -p "${DOCKER_CONFIG}"
chmod 700 "${DOCKER_CONFIG}"
login_succeeded=0
for attempt in 1 2 3; do
if echo "${DOCKERHUB_TOKEN}" | timeout 30s docker login docker.io --username "${DOCKERHUB_USERNAME}" --password-stdin; then
login_succeeded=1
break
fi
if [[ "$attempt" -lt 3 ]]; then
echo "::warning::Docker Hub login attempt ${attempt} failed; retrying."
sleep 5
fi
done
if [[ "$login_succeeded" -ne 1 ]]; then
echo "::warning::Docker Hub login failed after 3 attempts; continuing with anonymous pulls."
fi

- name: Set up Node
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.0.0
with:
node-version: 22
cache: npm

- name: Install root dependencies
run: npm ci --ignore-scripts

- name: Build CLI
run: npm run build:cli

- name: Install OpenShell CLI
run: bash scripts/install-openshell.sh

- name: Run upgrade stale sandbox live Vitest test
env:
NVIDIA_INFERENCE_API_KEY: ${{ secrets.NVIDIA_INFERENCE_API_KEY }}
run: |
set -euo pipefail
export PATH="$HOME/.local/bin:$HOME/.npm-global/bin:$PATH"
if command -v openshell >/dev/null 2>&1; then
OPENSHELL_BIN="$(command -v openshell)"
elif [ -x "$HOME/.local/bin/openshell" ]; then
OPENSHELL_BIN="$HOME/.local/bin/openshell"
else
echo "::error::OpenShell CLI not found after install"
ls -la /usr/local/bin/openshell "$HOME/.local/bin/openshell" 2>&1 || true
exit 1
fi
export OPENSHELL_BIN
"$OPENSHELL_BIN" --version
npx vitest run --project e2e-scenarios-live test/e2e-scenario/live/upgrade-stale-sandbox.test.ts --silent=false --reporter=default

- name: Upload upgrade stale sandbox artifacts
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-vitest-scenarios-upgrade-stale-sandbox
path: e2e-artifacts/vitest/upgrade-stale-sandbox/
include-hidden-files: false
if-no-files-found: ignore
retention-days: 14

- name: Clean up Docker auth
if: always()
run: |
set -euo pipefail
docker logout docker.io || true
rm -rf "${DOCKER_CONFIG}"

Comment on lines +1762 to +1868

@coderabbitai coderabbitai Bot Jun 18, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify the free-standing inventory includes the new job and scenario
echo "=== Checking free-standing inventory for upgrade-stale-sandbox ==="
if fd -t f 'free-standing-workflow-inventory' tools/; then
  inventory_file=$(fd -t f 'free-standing-workflow-inventory' tools/ | head -1)
  echo "Found inventory file: $inventory_file"
  echo ""
  echo "--- Checking for job id ---"
  rg -n 'upgrade-stale-sandbox-vitest' "$inventory_file" || echo "WARNING: Job not found in inventory"
  echo ""
  echo "--- Checking for scenario id ---"
  rg -n 'upgrade-stale-sandbox[^-]' "$inventory_file" || echo "WARNING: Scenario not found in inventory"
else
  echo "WARNING: Could not find free-standing-workflow-inventory file"
fi

Repository: NVIDIA/NemoClaw

Length of output: 402

The new job is well-structured, but the free-standing inventory file is incomplete.

The upgrade-stale-sandbox-vitest job correctly implements isolated Docker auth, retry logic, and artifact handling. However, verification confirms that tools/e2e-scenarios/free-standing-workflow-inventory.mts is missing the required entries:

  • upgrade-stale-sandbox-vitest not in allowed_jobs
  • upgrade-stale-sandbox not in free_standing_scenarios_csv
  • Mapping not in free_standing_scenario_jobs_csv

Since the workflow's generate-matrix step validates inventory coverage, these missing entries will cause CI failures. Add all three required entries to the inventory file before merging.

Minor style note: Line 1781 has excessive inline whitespace in the vitest command. Consider using backslash line continuation for consistency.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/e2e-vitest-scenarios.yaml around lines 1698 - 1799, The
free-standing workflow inventory file is missing three required entries that
will cause CI failures during the generate-matrix validation step. Open
tools/e2e-scenarios/free-standing-workflow-inventory.mts and add
upgrade-stale-sandbox-vitest to the allowed_jobs array, upgrade-stale-sandbox to
the free_standing_scenarios_csv string, and create the mapping entry in
free_standing_scenario_jobs_csv to connect the scenario to its job.
Additionally, on line 1781 in the upgrade-stale-sandbox-vitest job, the npx
vitest run command has excessive inline whitespace; refactor it to use backslash
line continuation for better readability and consistency with other multi-line
commands in the workflow.

double-onboard-vitest:
needs: generate-matrix
if: ${{ (inputs.jobs == '' && inputs.scenarios == '') || contains(format(',{0},', inputs.jobs), ',double-onboard-vitest,') || contains(format(',{0},', inputs.scenarios), ',double-onboard,') }}
Expand Down Expand Up @@ -3598,6 +3700,7 @@ jobs:
rebuild-openclaw-vitest,
sandbox-rebuild-vitest,
state-backup-restore-vitest,
upgrade-stale-sandbox-vitest,
token-rotation-vitest,
messaging-compatible-endpoint-vitest,
messaging-providers-vitest,
Expand Down
Loading
Loading