Skip to content

Report where trace capture time goes, behind DEVTOOLS_CAPTURE_TIMING - #519

Merged
vishnuv688 merged 1 commit into
mainfrom
perf/516-capture-timing
Nov 2, 2026
Merged

vishnuv688 merged 1 commit into
mainfrom
perf/516-capture-timing

Conversation

@vishnuv688

Copy link
Copy Markdown
Member

What & why

Part of #516 (step 1 of 2: measurement). Stacked on #518.

Trace mode is slower than live mode because of the per-action capture, and nothing measured where that time goes per platform, so any proposal to make capture cheaper was a guess. With DEVTOOLS_CAPTURE_TIMING=1, every capture is timed probe by probe and the run prints a summary at trace export, plus raw NDJSON records to share. It only measures: what is captured, probe order and concurrency are unchanged.

Type of change

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Polish (an improvement to an existing feature)
  • Breaking change (existing behavior changes for users)
  • Documentation
  • Internal (build, CI, dependencies, tooling)

Packages touched

  • shared (types and contracts)
  • core (framework-agnostic capture/reporting)
  • elements (published element/snapshot API — @wdio/elements)
  • service (WebdriverIO adapter)
  • nightwatch-devtools (Nightwatch adapter)
  • selenium-devtools (Selenium adapter)
  • selenium-devtools-py (Selenium Python adapter)
  • backend (server)
  • app (UI)
  • script (page-injected runtime)
  • trace (Trace mode)

Capability matrix

  • packages/conformance/src/features.ts updated, and tables regenerated with pnpm docs:capabilities, if this adds or changes user-facing behaviour

Not a capability: DEVTOOLS_CAPTURE_TIMING is listed in capability-options.ts as a diagnostic (notFeature).

Notes for reviewers

Why this is in core, though it touches every adapter: all three JS adapters capture through core's captureActionSnapshot, so the recording lives there, and reporting hooks into finalizeTraceExport. Each adapter only passes one optional argument: what triggered the capture. Python cannot import core, so it mirrors the recording and summary, and both summaries are checked against the same cases in mirror-vectors.json.

What a record holds: each probe's issue and settle times; its service time (settle minus the later of its own issue and the previous probe's settle), which separates a probe's own cost from time queued behind the driver's earlier requests; whether the read came back empty; the in-process build time (process, page-source parsing and locators on native); the whole capture; native or not; the command; and the trigger: an action, a wait's poll (Selenium, Nightwatch), a matcher's poll (WDIO, with captureAssertions on), or the final capture.

Off by default, at no cost: with the variable unset, no timer is created and timed() returns the probe's own promise. Any value other than empty, 0, false, no or off turns it on, as Python's existing DEVTOOLS_* switches read.

Sample output (from a unit-test fixture, not a real run)

[devtools] capture timing: 4 captures (0 native; action 2, wait-poll 1, matcher-poll 1), busy 1.6 s of 10.0 s wall (15.8%)
probe (ms)  n  total  median   p95   max  svc total  svc median
url         4     16       4     4     4         16           4
screenshot  4   1616     254  1004  1004       1600         250
capture     4   1640     260  1010  1010       1640         260

Testing

  • pnpm install --frozen-lockfile, pnpm build, pnpm test (3411 passed) and pnpm lint are green; the Python suite passes (849 tests) and its change fragment is present; no workspace import leaks in the adapter bundles.
  • New tests: recording, service-time and summary math, off by default, the env flag parsing, and the shared mirror vectors in both languages.

Next (step 2 of #516): run the commands in the README's new "Measuring capture cost" section on each platform and post the numbers to #516, before proposing any change to capture. Only Selenium's example switches browser (SELENIUM_BROWSER); the other desktop examples run Chrome.

Not in this PR: the docs-site mirror of the README section, with the other docs-site changes once trace.webdriver.io is live.

Screenshots / recordings

Not applicable: no UI change.

🤖 Generated with Claude Code

Trace mode is slower than live mode because of the per-action capture, and
nothing measured where that time goes per platform, so any proposal to make
capture cheaper was a guess (#516). With DEVTOOLS_CAPTURE_TIMING set, every
capture records each probe's issue and settle times, its service time apart
from the time it queued behind the driver's earlier requests, and what
triggered it (an action, a wait's poll, a matcher's poll). At trace export
the run prints a summary per probe and writes the raw records as NDJSON.

It only measures: probe order, concurrency and what is captured are
unchanged, and with the variable unset no timer is created. All three JS
adapters get it from core; Python mirrors it, with the summary checked by
shared mirror vectors.

Refs #516

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vishnuv688
vishnuv688 force-pushed the ci/511-hosted-player-e2e branch from 357cead to dcf13e9 Compare October 30, 2026 10:12
@vishnuv688
vishnuv688 force-pushed the perf/516-capture-timing branch from 9314b3f to 09ddb75 Compare October 30, 2026 10:13
@vishnuv688
vishnuv688 changed the base branch from ci/511-hosted-player-e2e to main November 2, 2026 10:02
@vishnuv688
vishnuv688 merged commit 3251f03 into main Nov 2, 2026
10 checks passed
@vishnuv688
vishnuv688 deleted the perf/516-capture-timing branch November 2, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant