Merge pull request #16 from pavelevgrafov/ci-selftest-rename #41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: Design-Ops Heavy Floor | |
| # What CI checks here is the TOOLKIT, not a product. Running `dops verify` | |
| # against this repository's own root asks "does the toolkit pass the floor it | |
| # applies to sites?" — and the answer is meaninglessly no: there is no | |
| # skeleton, no tokens and no contract to verify. The first live run failed | |
| # exactly that way (D16: no not_approved_visual_design marker). | |
| # | |
| # So: the self-test is the toolkit's acceptance, and the end-to-end job | |
| # materialises a real project from a starter and runs the floor against THAT. | |
| # Feature branches are checked through their pull request, which is what the | |
| # branch protection on main actually requires; listing them here as well only | |
| # doubled every run. `push` stays for main itself. | |
| on: | |
| push: | |
| branches: [main] | |
| pull_request: | |
| branches: [main] | |
| schedule: | |
| # Nightly: the end-to-end job with the browser lane | |
| - cron: "0 3 * * *" | |
| jobs: | |
| selftest: | |
| # Was "Self-test (fs-level)" until the browser probes started running here | |
| # too, at which point the name described half the job. Renaming it is two | |
| # changes that must happen together — this file and the branch-protection | |
| # rule, which requires the check name as a literal string. Change one and | |
| # every pull request waits forever for a check that will never report | |
| # under the name the rule is watching for. | |
| name: Self-test | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.11" | |
| - uses: actions/setup-node@v4 | |
| with: | |
| node-version: "20" | |
| # The browser half of the suite — D22 visual regression, the П-4 panel, | |
| # the П-6 feed, the Ф-1 form and its overlap probe — is guarded by | |
| # `require.resolve('playwright')` and had NEVER run here: this job had | |
| # no node, so it printed one honest `skip` line and moved on. The line | |
| # was true and nobody read it as "every UI acceptance probe is | |
| # unverified". That is the fourth time in this project that a check | |
| # existed and was not the one that ran, and the reason is always the | |
| # same shape — a capability the environment silently lacked. | |
| # | |
| # Installing it here is what makes those probes probes. The extra | |
| # minute buys the difference between a suite that passes and a suite | |
| # that ran. | |
| - name: Install dependencies | |
| run: | | |
| python3 -m pip install --quiet pyyaml | |
| npm install -D playwright axe-core | |
| npx playwright install --with-deps chromium | |
| - name: Tooling acceptance | |
| run: bash tools/dops selftest | |
| - name: Package self-test | |
| run: bash eval/selftest/run-self-test.sh | |
| end-to-end: | |
| name: End-to-end floor on a real build | |
| # Runs on every event, including push. It used to skip on push, which was | |
| # cheap and quietly dangerous: a skipped job still publishes a check run | |
| # under the same name, so a branch protection rule requiring this check | |
| # could be satisfied by the skip instead of by the real run. A required | |
| # check that a skip can satisfy is not a gate. One extra minute per push | |
| # is the price of the gate meaning what it says. | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v5 | |
| with: | |
| python-version: "3.11" | |
| - uses: actions/setup-node@v4 | |
| with: | |
| node-version: "20" | |
| - name: Install dependencies | |
| run: | | |
| python3 -m pip install --quiet pyyaml | |
| npm install -D playwright axe-core | |
| npx playwright install --with-deps chromium | |
| # The rehearsal itself is NOT written here. It lives in | |
| # eval/e2e/rehearse.sh, which this job and the implementer both run, so | |
| # "it passes locally" and "it passes in CI" cannot mean two different | |
| # things. They did once: a hand-retyped local copy carried three token | |
| # files where this job carried two, and the difference hid a D.41 defect | |
| # until it turned main red (bridge protocol, amendment 07 §2). | |
| # | |
| # Adding a rehearsal step back into this file is a defect, not a | |
| # shortcut — eval/selftest/run-self-test.sh fails the build for it. | |
| - name: End-to-end rehearsal | |
| run: bash eval/e2e/rehearse.sh --root /tmp/e2e --out /tmp/floor.json --keep | |
| - name: Upload floor report | |
| if: always() | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: floor-report | |
| path: | | |
| /tmp/floor.json | |
| /tmp/e2e/artifacts/audit/shots-manifest.json |