Skip to content

Commit 0883c25

Browse files
vishnuv688claude
andcommitted
Move the trace zip reader from backend into trace
The planned trace query MCP server (#493) needs to parse trace zips, and depending on backend for that would pull Fastify, the dashboard bundle and the rerun spawner into its install. The reader imports only shared and trace, so it moves to trace, which now owns the format in both directions. readTraceZip moves with parseTraceZip because trace already does the matching disk write and both backend and the MCP package read from disk. The reader stays out of trace's index, so core and the adapters never inline it. No behaviour change: backend imports it by subpath. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent c60530c commit 0883c25

19 files changed

Lines changed: 26 additions & 36 deletions

‎ARCHITECTURE.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ A devtools dashboard for end-to-end browser tests. Four adapters — WebdriverIO
1919
│ │
2020
│ ▼
2121
│ [trace] ◀── also used by backend
22-
│ pure transforms: events → trace.zip
22+
│ pure transforms: events ⇄ trace.zip
2323
▼ (WS frames typed by shared)
2424
[backend] Fastify + WS gateway + baseline store + rerun spawner
2525
│
@@ -55,9 +55,9 @@ Imports from: nothing. Imported by: every other package.
5555

5656
### `packages/trace`
5757

58-
Trace-format transforms: captured events in, `trace.zip` out. Workspace-internal; inlined into each consumer at build time.
58+
Trace-format transforms: captured events in, `trace.zip` out, and a `trace.zip` back into a player payload. Workspace-internal; inlined into each consumer at build time.
5959

60-
Split out of `core` because **two layers need it and only one of them may import `core`**. Adapters build their own trace through `core`; the backend builds one on behalf of an adapter that cannot run Node-side trace code (the Python adapter), and §2.2 bars the backend from `core`. Everything here is a pure transform over `shared` types — no framework API, no driver, no capture, no I/O beyond writing the zip.
60+
Split out of `core` because **two layers need it and only one of them may import `core`**. Adapters build their own trace through `core`; the backend builds one on behalf of an adapter that cannot run Node-side trace code (the Python adapter), and §2.2 bars the backend from `core`. Everything here is a pure transform over `shared` types — no framework API, no driver, no capture, no I/O beyond writing and reading the zip.
6161

6262
Contains:
6363

@@ -67,6 +67,7 @@ Contains:
6767
- `buildImageFrameSnapshots` / `upsertRichestSnapshot` / `buildDenseScreencast` / `thinScreencastFrames` — filmstrip and per-action frame snapshots.
6868
- `buildMutationsNdjson` / `reattributeDomAnchors` — the DOM mutation stream and its anchor repositioning.
6969
- `buildConsoleEvents`, `networkRequestToHar`, `buildSourceResources` / `sourceResourceName` / `callSourceToStack`, `generateTranscript`, `sha1Hex`.
70+
- `parseTraceZip` / `readTraceZip` (`trace-reader.ts` and its `trace-reader-*` siblings) — the reverse direction: a `trace.zip`, ours or a foreign one, back into the `TracePlayerData` the player renders. Imported by subpath (`@wdio/devtools-trace/trace-reader`) and kept out of the package index, so adapters that reach `trace` through `core` never inline it.
7071

7172
Imports from: `shared`. Imported by: `core` (which re-exports it, so adapters reach it unchanged) and `backend`.
7273

@@ -274,7 +275,7 @@ The repo has converged on a clear ownership story. When in doubt, the top-down d
274275

275276
- A type, constant, enum, schema, or contract used by more than one package → **`shared`**.
276277
- Capture, parsing, normalization, sourcemap, UID, reporter, screencast, or WS-framing logic that doesn't depend on a specific framework's API → **`core`**.
277-
- A pure transform that turns captured events into trace-zip content → **`trace`**. The test is whether the backend would ever need it: it builds traces for adapters that can't, and it may not import `core`.
278+
- A pure transform that turns captured events into trace-zip content, or a zip back into a player payload → **`trace`**. The test is whether the backend would ever need it: it builds traces for adapters that can't, and it may not import `core`.
278279
- A specific framework's hook, driver patch, or runner integration → the matching **adapter** package. Adapter code calls `core` for the actual work and only owns the hook registration.
279280
- A backend HTTP route, WS handler, or rerun behavior → **`backend`**, with the contract added to `shared` first.
280281
- UI → **`app`**, consuming `shared` contracts only.

‎CLAUDE.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,11 +81,11 @@ Some helpers are framework-agnostic by nature but used in only one adapter today
8181

8282
### Trace-format transforms live in `trace`, one layer below `core`
8383

84-
`packages/trace` holds the pure transforms that turn captured events into trace-zip content — the zip writer, action events, group paths, frame snapshots, mutations, HAR, sources, transcript. `core` keeps the adapter-side *orchestration and policy* that calls them: `trace-finalizer`, `spec-trace-helpers`, `trace-retention`.
84+
`packages/trace` holds the pure transforms that turn captured events into trace-zip content — the zip writer, action events, group paths, frame snapshots, mutations, HAR, sources, transcript — and the reader that turns a zip back into a player payload (`trace-reader.ts`), so the format has one owner in both directions. `core` keeps the adapter-side *orchestration and policy* that calls them: `trace-finalizer`, `spec-trace-helpers`, `trace-retention`.
8585

8686
The split is not aesthetic. **`backend` may not import `core`** (it would pull framework-adapter logic into the server), but it does need to build a trace on behalf of an adapter that can't — the Python adapter ships no Node. `trace` is the layer both can reach, so it may import `shared` and nothing else; a single import of `core` from it re-creates the cycle the split exists to remove, and ESLint enforces that.
8787

88-
The test for a new helper: *would the backend ever need this to build a zip?* If yes, `trace`. If it needs a driver, a framework hook, or a capture session, `core`.
88+
The test for a new helper: *would the backend ever need this to build or read a zip?* If yes, `trace`. If it needs a driver, a framework hook, or a capture session, `core`.
8989

9090
### One DOM snapshot per action, taken before it
9191

‎eslint.config.cjs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -330,7 +330,7 @@ module.exports = [
330330
{
331331
group: ['@wdio/devtools-trace', '@wdio/devtools-trace/*'],
332332
message:
333-
'App must not import from trace (CLAUDE.md §2.2). trace WRITES the zip; the app reads one the backend has already parsed for it.'
333+
'App must not import from trace (CLAUDE.md §2.2). The backend writes and parses zips with it; the app receives a parsed trace over HTTP.'
334334
}
335335
]
336336
}

‎packages/backend/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ The server that the three adapter packages connect to and the dashboard UI talks
99
- **Baseline store** (in-memory) — captures a snapshot of a failing test attempt, plus per-uid metadata, so the "Preserve & Rerun" flow can show a side-by-side diff.
1010
- **Rerun spawner** (`runner.ts`) — spawns the user's `wdio` / `nightwatch` / `selenium` binary with rerun filters built from the dashboard's payload.
1111
- **Worker-message handler** — dispatches messages from spawned workers (config path, session id, video path, ...).
12-
- **Trace serving** (`show-trace.ts`, `trace-reader.ts`) — reconstructs a recorded `trace.zip` and boots the server in a read-only **trace-serve** mode that backs the DevTools UI's trace player. No worker connects in this mode.
12+
- **Trace serving** (`show-trace.ts`, with the reader from `@wdio/devtools-trace`) — reconstructs a recorded `trace.zip` and boots the server in a read-only **trace-serve** mode that backs the DevTools UI's trace player. No worker connects in this mode.
1313

1414
## Framework awareness
1515

@@ -27,7 +27,7 @@ Trace mode (see the [root README](../../README.md#-trace-mode-tracezip)) writes
2727

2828
The same entry is shipped as a `show-trace` **bin** by the backend (`./dist/show-trace.js`) and by each adapter (`@wdio/devtools-service`, `@wdio/selenium-devtools`, `@wdio/nightwatch-devtools` each ship a thin `bin/show-trace.mjs`), so `npx show-trace <trace.zip>` works in any project that installs an adapter — backend need only be a transitive dependency. In trace-serve mode `start({ trace })` exposes the reconstructed payload at `TRACE_API.get` and skips the worker/rerun machinery. It is one of the backend's two bins; the other, `devtools-backend` (`./dist/server.js`), starts the live dashboard server described under [Public API](#public-api).
2929

30-
- **Reader** (`src/trace-reader.ts`, with sibling `trace-reader-{constants,types,utils,groups}.ts`) — `parseTraceZip` / `readTraceZip` reconstruct a `TracePlayerData` payload from the archive. It accepts this repo's own exporter output **and foreign zips** written by other tools (every `.trace` entry is an action-event stream, every `.network` a HAR stream, `.stacks` sidecars carry call stacks). It rebuilds:
30+
- **Reader** (`packages/trace/src/trace-reader.ts`, with sibling `trace-reader-{constants,types,utils,groups}.ts`) — `parseTraceZip` / `readTraceZip` reconstruct a `TracePlayerData` payload from the archive. It accepts this repo's own exporter output **and foreign zips** written by other tools (every `.trace` entry is an action-event stream, every `.network` a HAR stream, `.stacks` sidecars carry call stacks). It rebuilds:
3131
- **commands** — from `before`/`after` action events, with call source, result, error, nearest screenshot frame, and pointer hit point;
3232
- **DOM mutations** — from the `.mutations` NDJSON stream (drives the player's DOM time-travel);
3333
- **network** — HAR entries from `.network` streams;

‎packages/backend/src/show-trace.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ import path from 'node:path'
99
import process from 'node:process'
1010
import { fileURLToPath } from 'node:url'
1111

12-
import { readTraceZip } from './trace-reader.js'
12+
import { readTraceZip } from '@wdio/devtools-trace/trace-reader'
1313
import { start } from './index.js'
1414

1515
const USAGE = `Usage: show-trace <trace.zip>

‎packages/shared/src/trace-file.ts‎

Lines changed: 1 addition & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,4 @@
1-
// The trace.zip file layout — the one definition of how a trace is packaged.
2-
//
3-
// `core` writes these entries and `backend` reads them, and the two packages
4-
// cannot import each other (backend depends on shared only). Before this module
5-
// both sides carried their own copies of these strings, so a rename in the
6-
// writer left every reader test green and the mismatch only surfaced when
7-
// someone opened a real trace. Import from here on both sides instead.
8-
//
9-
// The *event interfaces* deliberately stay separate: `core`'s are strict (the
10-
// writer always emits `pageId`, `apiName`, `params`, `title`) while the reader's
11-
// are permissive, because it also parses traces this repo did not write — see
12-
// `backend/src/trace-reader-types.ts`. Strict producer, lenient consumer; only
13-
// the names below have to agree.
1+
// The trace.zip entry names, shared by the writer and the reader; their event types stay separate because the reader also parses foreign zips.
142

153
/** Entry names inside a trace.zip, and the directory prefix for blobs. Also the
164
* filenames used by the unpacked `ndjson-directory` format. */

‎packages/trace/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
"version": "1.0.0",
44
"private": true,
55
"author": "Vishnu Vardhan",
6-
"description": "Trace-format transforms — turns captured events into the trace zip. Split out of core so the backend can build a trace for adapters that cannot (CLAUDE.md §2.2 bars backend from core). Workspace-internal, never published — code is inlined into each consuming package at build time.",
6+
"description": "Trace-format transforms — turns captured events into the trace zip and reads a zip back into a player payload. Split out of core so the backend can build a trace for adapters that cannot (CLAUDE.md §2.2 bars backend from core). Workspace-internal, never published — code is inlined into each consuming package at build time.",
77
"repository": {
88
"type": "git",
99
"url": "git+https://github.com/webdriverio/devtools.git",

‎packages/trace/src/index.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
// Trace-format transforms: captured events in, trace zip out. See
1+
// Trace-format transforms: captured events in, trace zip out. The reader
2+
// (`trace-reader.ts`) is left out of this index so `core` never inlines it. See
23
// ARCHITECTURE.md §2 and CLAUDE.md §2.2.
34
//
45
// Split out of `core` because two layers need it and `core` is reachable from
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)