Refine reasoner regression assert rtol & refine workflow setting #7
Workflow file for this run
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | |
| # SPDX-License-Identifier: OpenMDW-1.1 | |
| # Generator (VFM) SFT loss regression on a self-hosted 8×H200 runner (4-GPU | |
| # subset). Runs the single ``vision_sft_nano`` spec of | |
| # tests/launch_regression_test.py. | |
| # | |
| # Requires: | |
| # * a self-hosted runner labelled [self-hosted, gpu, h200] with >=4 GPUs, | |
| # NVIDIA drivers, and `uv` on PATH; | |
| # * an `HF_TOKEN` repository secret (gated dataset/model downloads). | |
| # | |
| # The h100 goldens are reused on H200 (see _detect_arch). | |
| name: GPU Regression (Generator) | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| branches: [main] | |
| concurrency: | |
| group: gpu-regression-generator-${{ github.ref }} | |
| cancel-in-progress: true | |
| jobs: | |
| generator-regression: | |
| runs-on: [self-hosted, gpu, h200] | |
| timeout-minutes: 60 | |
| env: | |
| HF_TOKEN: ${{ secrets.HF_TOKEN }} | |
| HF_HUB_DISABLE_XET: "1" | |
| # Select the 4-GPU regression test variant (uses 4 of the 8 GPUs). | |
| TEST_MAX_GPUS: "4" | |
| steps: | |
| - uses: actions/checkout@v6 | |
| - uses: astral-sh/setup-uv@v7 | |
| - name: Sync environment (cu128-train) | |
| run: uv sync --all-extras --group=cu128-train | |
| # Generator (vision_sft_nano) loss vs the h100 goldens. -s streams the live log. | |
| - name: Generator regression (vision_sft_nano, 4-GPU subset) | |
| run: | | |
| export LD_LIBRARY_PATH= | |
| uv run --all-extras --group=cu128-train python -m pytest -v -s \ | |
| tests/launch_regression_test.py -k vision_sft_nano \ | |
| --num-gpus=4 --levels=2 -o addopts= | |
| # The h100_inputs fixture removes its DCP stage on teardown; clear the | |
| # pytest tmp dirs too (logs + any run output). The HF cache is kept. | |
| - name: Clean up run outputs | |
| if: always() | |
| run: | | |
| rm -rf "${TMPDIR:-/tmp}"/pytest-of-* /tmp/pytest-of-* || true |