Skip to content

Test the trace reader against real zips from an older writer - #515

Merged
vishnuv688 merged 1 commit into
mainfrom
test/510-trace-format-fixtures
Nov 2, 2026
Merged

vishnuv688 merged 1 commit into
mainfrom
test/510-trace-format-fixtures

Conversation

@vishnuv688

@vishnuv688 vishnuv688 commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

What & why

Part of #510 and #494. Stacked on #514.

The hosted player always runs the newest reader, while people open zips written by whatever adapter version recorded them. This commits four real trace zips, one per adapter, from the writer as it was before #508, with a test that the current reader still yields what each one held. A reader change that breaks an older trace now fails pnpm test.

Type of change

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Polish (an improvement to an existing feature)
  • Breaking change (existing behavior changes for users)
  • Documentation
  • Internal (build, CI, dependencies, tooling)

Packages touched

  • shared (types and contracts)
  • core (framework-agnostic capture/reporting)
  • elements (published element/snapshot API — @wdio/elements)
  • service (WebdriverIO adapter)
  • nightwatch-devtools (Nightwatch adapter)
  • selenium-devtools (Selenium adapter)
  • selenium-devtools-py (Selenium Python adapter)
  • backend (server)
  • app (UI)
  • script (page-injected runtime)
  • trace (Trace mode)

Capability matrix

  • packages/conformance/src/features.ts updated, and tables regenerated with pnpm docs:capabilities, if this adds or changes user-facing behaviour

Not needed: tests only.

Notes for reviewers

Fixtures

Fixture Original Committed Covers
wdio-mocha-2026-10-28-14977b93 261,400 B 19,762 B the 13-action spec
selenium-mocha-2026-10-28-14977b93 159,706 B 19,098 B a failed wait, a failed node:assert, JPEG screencast frames beside PNG ones
nightwatch-bdd-2026-10-28-14977b93 376,379 B 29,946 B waitForElementVisible rows, a failed native assert
python-pytest-2026-10-28-14977b93 804,136 B 55,080 B three browser sessions in one zip

From the examples-smoke artifact of CI run 40774299701 (commit 14977b93, 2026-10-28). The two earlier artifacts available wrote the same format, so they add nothing a reader could break separately. The whole fixture directory is 164 KB.

Reduction

  • reduce-fixture.ts, committed beside the fixtures so the next one is made the same way, keeps every entry, in order and under its name, and changes only images. Each image resource becomes a 1x1 image of the same format carrying its own resource name; each inline data:image payload in .trace streams becomes a 1x1 image; every other entry keeps its bytes. Reducing the same source twice gives byte-identical zips.
  • This does not weaken the test: the reader never decodes an image. The test reads each frame's embedded resource name back, so a frame paired with the wrong resource still fails.
  • Each manifest is counted from the zip's raw entries, not through the reader, so a reader bug cannot write itself into the expected values. Breaking the reader on purpose (dropping network entries, reversing frame order) failed seven of the tests.

Sensitive data

Checked every non-image entry: only CI runner paths, the local fixture host and the-internet's public demo credentials. The original screenshots are gone.

Why this leaves #510 open

These cover one format: the one written until #508. Still open in #510: a fixture for each later format change as it ships, and a foreign trace, for which no zip was available.

Testing

pnpm build, pnpm test (3337 passed) and pnpm lint are green, on top of #508 and #513, so the current reader opens zips written before both.

Screenshots / recordings

Not applicable: no UI change.

🤖 Generated with Claude Code

The hosted player always runs the newest reader, while people open zips
whatever adapter version recorded. Four real traces, one per adapter, from
the writer as it was before the screenshot dedupe, are committed with a
manifest counted from their raw entries, and a test checks the current
reader still yields what each held.

Real zips are mostly screenshots, so a committed script reduces each one:
images become 1x1 images of the same format carrying their own resource
name, and every other entry keeps its bytes. The reader never decodes an
image, so nothing it can get wrong is lost; the set is 164 KB against
1.6 MB.

Refs #510

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vishnuv688
vishnuv688 force-pushed the perf/497-parse-in-worker branch from 6bfa403 to 938e36a Compare October 30, 2026 10:12
@vishnuv688
vishnuv688 force-pushed the test/510-trace-format-fixtures branch from 85c12cd to 857f076 Compare October 30, 2026 10:12
@vishnuv688
vishnuv688 changed the base branch from perf/497-parse-in-worker to main November 2, 2026 09:25
@vishnuv688
vishnuv688 merged commit 23d8bf2 into main Nov 2, 2026
7 checks passed
@vishnuv688
vishnuv688 deleted the test/510-trace-format-fixtures branch November 2, 2026 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant