Skip to content

Check the hosted trace player end to end on every PR - #518

Merged
vishnuv688 merged 2 commits into
mainfrom
ci/511-hosted-player-e2e
Nov 2, 2026
Merged

vishnuv688 merged 2 commits into
mainfrom
ci/511-hosted-player-e2e

Conversation

@vishnuv688

Copy link
Copy Markdown
Member

What & why

Closes #511. Part of #494. Stacked on #517.

The hosted trace player was only ever checked by hand. A new CI job builds it, serves it under the headers vercel.json sends, and drives it with WebdriverIO in headless Chrome on every PR. The same check runs on a schedule and after each production deploy against the deployed page, and keeps one issue open while it fails.

Type of change

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Polish (an improvement to an existing feature)
  • Breaking change (existing behavior changes for users)
  • Documentation
  • Internal (build, CI, dependencies, tooling)

Packages touched

  • shared (types and contracts)
  • core (framework-agnostic capture/reporting)
  • elements (published element/snapshot API — @wdio/elements)
  • service (WebdriverIO adapter)
  • nightwatch-devtools (Nightwatch adapter)
  • selenium-devtools (Selenium adapter)
  • selenium-devtools-py (Selenium Python adapter)
  • backend (server)
  • app (UI)
  • script (page-injected runtime)
  • trace (Trace mode)

Only conformance (CI checks), which the list does not include, plus the workflows.

Capability matrix

  • packages/conformance/src/features.ts updated, and tables regenerated with pnpm docs:capabilities, if this adds or changes user-facing behaviour

Not needed: CI only.

Notes for reviewers

What the check does (pnpm player:e2e, code in packages/conformance/hosted-player/)

  1. Fetches the page and fails unless it carries script-src 'self' and connect-src 'self', so it can never pass against a page without the policy.
  2. Opens each committed older-format fixture (Test the trace reader against real zips from an older writer #515) and a zip written by this checkout's writer, through the landing screen's file input (an explicit shadow$ chain, then setFiles).
  3. Expands the action tree and compares the rendered row labels with the fixture's manifest; requires a filmstrip frame and no error banner.
  4. Opens a non-zip and expects the "Could not open" error.
  5. Fails on any console error, collected through BiDi log.entryAdded. CSP violations are not BiDi log events, so a preload script turns each securitypolicyviolation into a console error.
  6. Requires a request for /assets/trace-parse.worker-*.js, from BiDi network events or Resource Timing, proving the Web Worker path (Parse the hosted player's trace in a Web Worker #514) under the real CSP.

Design notes

  • It lives in conformance rather than app: the app may not import the trace writer, which the check needs, and conformance already holds the smoke job's checks and depends on webdriverio and trace.
  • A standalone remote() script rather than the testrunner: one linear check that also starts and stops the preview server.
  • Its own job, because it needs only the app's hosted build, so it runs beside the other jobs; Chrome is the runner image's, and WebdriverIO fetches the matching driver.
  • Scheduled workflow (scheduled-hosted-player.yml): every 6 hours, on workflow_dispatch, and on a successful Production deployment_status from Vercel. Skipped while vars.HOSTED_PLAYER_URL is unset. contents: read, with issues: write only on the report job; one run at a time; comments on the open "Hosted trace player check is failing" issue or opens it, and closes it when the check passes. The URL reaches scripts only through env. The bot-opened issue has no assignee.

Testing

  • pnpm build, pnpm test (3373 passed), pnpm lint green; pnpm install --frozen-lockfile passes.
  • 35 unit tests for the non-browser parts: argument parsing, manifest reading and label diffing, the CSP header check, the log classifier, the worker URL matcher, the page scripts under happy-dom, and the current-writer zip read back through the reader.
  • This PR's own hosted-player job is the first real run. Not run locally: real-browser runs are left to CI.

Proven only by CI

  • setFiles through the shadow$ chain; BiDi log and preload capture of real errors; the worker request showing up; labels matching the real UI; the timeouts.
  • The scheduled workflow is reviewed, not executed, until HOSTED_PLAYER_URL is set.

Screenshots / recordings

Not applicable: no UI change.

🤖 Generated with Claude Code

The hosted player was only ever checked by hand. A new CI job builds it,
serves it under the headers vercel.json sends, and in headless Chrome opens
the committed older-format fixtures, a zip from this checkout's writer and
a non-zip through it. It fails on a missing CSP, on rows that do not match
the trace, on a missing filmstrip, and on any console error or CSP
violation, and it checks the trace was parsed by the Web Worker.

The same check runs on a schedule and after each production deploy against
vars.HOSTED_PLAYER_URL, skipped while that is unset, and keeps one issue
open while it fails.

Closes #511

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vishnuv688
vishnuv688 force-pushed the fix/504-sidebar-background branch from 831b6a1 to 8211eee Compare October 30, 2026 10:12
@vishnuv688
vishnuv688 force-pushed the ci/511-hosted-player-e2e branch from 357cead to dcf13e9 Compare October 30, 2026 10:12
@vishnuv688
vishnuv688 changed the base branch from fix/504-sidebar-background to main November 2, 2026 09:34
main's Dependabot vite bump (#449) replaced every vite 8.3.2 lockfile entry
with 8.3.5, and this branch's conformance importer still named 8.3.2, so the
merged lockfile failed a frozen install. Its entry now names main's own vite
8.3.5 snapshot. Only those three lines differ from main's lockfile: a full
regeneration also re-resolved unrelated peers across 1,200 lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vishnuv688
vishnuv688 merged commit fd46bee into main Nov 2, 2026
12 checks passed
@vishnuv688
vishnuv688 deleted the ci/511-hosted-player-e2e branch November 2, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hosted player: scheduled check that trace.webdriver.io opens a trace

1 participant