Skip to content

Commit 7a751ea

Browse files
vishnuv688claude
andcommitted
Name trace screenshots by their actual image format
Every screenshot resource was written as .jpeg, though WebDriver and the polling recorder capture PNG, so a viewer that takes the content type from the file name served a PNG as image/jpeg. The name now follows the bytes, for the per-action screenshots and the dense filmstrip alike, and the resource, filmstrip frame, frame-snapshot override and synthetic resource all take it from one function. The reader opens whatever name an event records, so zips written before this still open. Closes #505 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent bd95ccb commit 7a751ea

11 files changed

Lines changed: 178 additions & 22 deletions
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
'@wdio/devtools-backend': patch
3+
'@wdio/devtools-service': patch
4+
'@wdio/selenium-devtools': patch
5+
'@wdio/nightwatch-devtools': patch
6+
---
7+
8+
A trace's screenshots are named by their actual image format. A PNG capture used to be stored as `resources/<page>-<timestamp>.jpeg`, so a viewer that takes the content type from the file name served it as `image/jpeg`; it is now `.png`, and so is a PNG frame of the dense filmstrip. Traces written before this change still open.

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -225,7 +225,7 @@ In trace mode no DevTools UI window opens. At session end the adapter writes tra
225225
The trace artifact contains:
226226
- `trace.trace` — NDJSON `context-options` + `before`/`after` action events. When test hooks are available (Mocha's `it()` / Cucumber's `Scenario()`), each test becomes a `Tracing.tracingGroup` span — an open/close `before`/`after` pair with `method: "tracingGroup"` and `params.name` set to the test title. Child actions inside the group carry `parentId` pointing back to the group's `callId`, so timeline viewers render tests as labelled spans wrapping their commands. For viewers that render a DOM snapshot per action, it also carries one `frame-snapshot` event per action: a page holding just that action's screenshot. The image is not stored a second time: its `src` is a placeholder `https://screenshot.wdio-devtools.invalid/` address, which the event's `resourceOverrides` maps to the screenshot's file under `resources/`.
227227
- `trace.network` — HAR-style network entries derived from the existing capture
228-
- `resources/page@<id>-<ts>.jpeg` — screenshot per user-facing action
228+
- `resources/page@<id>-<ts>.png` or `.jpeg`: screenshot per user-facing action, named by its actual image format
229229
- `resources/page@<id>-<ts>-elements.json` — flat interactable element list extracted by the page-injected scripts in `@wdio/devtools-core/element-scripts`
230230
- `resources/page@<id>-<ts>-snapshot.txt` — depth-indented accessibility-tree snapshot (AI-friendly)
231231
- `transcript.md` — human/LLM-readable Markdown transcript of the captured actions, with timing, selectors, and value annotations

‎packages/shared/src/image.ts‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,17 @@ export interface ImageSize {
2121
height: number
2222
}
2323

24-
/** Detect image mime from a base64 string's magic bytes — trace screenshots may
25-
* be PNG (polling capture) or JPEG (CDP), and the zip names both `.jpeg`. */
24+
/** Detect image mime from a base64 string's magic bytes: trace screenshots may
25+
* be PNG (polling capture) or JPEG (CDP). */
2626
export function imageMime(base64: string): string {
2727
return base64.startsWith(PNG_BASE64_MAGIC) ? 'image/png' : 'image/jpeg'
2828
}
2929

30+
/** File extension, without the dot, naming a base64 image's own format. */
31+
export function imageExtension(base64: string): string {
32+
return imageMime(base64).slice('image/'.length)
33+
}
34+
3035
/** Intrinsic pixel size read out of a base64 PNG or JPEG header, or null when
3136
* the bytes name none. A capture is fitted by this rather than by the metadata
3237
* viewport: the two disagree on both mobile platforms — Android reports the

‎packages/shared/src/trace-player.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,11 @@ export const TRACE_API = {
1010
} as const
1111

1212
/** A single screenshot frame reconstructed from a trace.zip's
13-
* `screencast-frame` events + `resources/*.jpeg`. */
13+
* `screencast-frame` events + their `resources/` images. */
1414
export interface TracePlayerFrame {
1515
/** Absolute wall-clock ms when the frame was captured. */
1616
timestamp: number
17-
/** Base64-encoded JPEG (no `data:` prefix), matching `CommandLog.screenshot`. */
17+
/** Base64-encoded PNG or JPEG (no `data:` prefix), matching `CommandLog.screenshot`. */
1818
screenshot: string
1919
}
2020

‎packages/shared/tests/image.test.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { describe, expect, it } from 'vitest'
22

3-
import { imageDimensions, imageMime } from '../src/image.js'
3+
import { imageDimensions, imageExtension, imageMime } from '../src/image.js'
44

55
/** 12x26 and 32x20 RGB PNGs — asymmetric so a swapped axis fails the test. */
66
const PORTRAIT_PNG =
@@ -46,6 +46,13 @@ describe('imageMime', () => {
4646
})
4747
})
4848

49+
describe('imageExtension', () => {
50+
it('names the extension after the sniffed format', () => {
51+
expect(imageExtension(PORTRAIT_PNG)).toBe('png')
52+
expect(imageExtension(PORTRAIT_JPEG)).toBe('jpeg')
53+
})
54+
})
55+
4956
describe('imageDimensions', () => {
5057
it("reads a PNG's IHDR", () => {
5158
expect(imageDimensions(PORTRAIT_PNG)).toEqual({ width: 12, height: 26 })

‎packages/trace/src/screencast-trace.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
*/
1414

1515
import { createHash } from 'node:crypto'
16-
import type { ScreencastFrame } from '@wdio/devtools-shared'
16+
import { imageExtension, type ScreencastFrame } from '@wdio/devtools-shared'
1717
import type { TraceZipResource } from './trace-zip-writer.js'
1818
import type { ScreencastFrameEvent } from './trace-snapshots.js'
1919

@@ -112,7 +112,7 @@ export function buildDenseScreencast(
112112
}
113113
const data = Buffer.from(frame.data, 'base64')
114114
const sha1 = createHash('sha1').update(data).digest('hex')
115-
const resourceName = `${sha1}.jpeg`
115+
const resourceName = `${sha1}.${imageExtension(frame.data)}`
116116
if (!seen.has(sha1)) {
117117
seen.add(sha1)
118118
resources.push({ resourceName, data })

‎packages/trace/src/trace-frame-snapshots.ts‎

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,11 @@
11
// Image-backed frame-snapshots, a shim until real DOM snapshots are captured: each action is a page
22
// whose <img> resolves, through `resourceOverrides`, to the screenshot already stored under `resources/`.
33

4-
import { imageMime, type ActionSnapshot } from '@wdio/devtools-shared'
4+
import {
5+
imageExtension,
6+
imageMime,
7+
type ActionSnapshot
8+
} from '@wdio/devtools-shared'
59
import { networkRequestToHar, type ResourceSnapshotEntry } from './trace-har.js'
610
import { screenshotResourceName } from './trace-snapshots.js'
711

@@ -149,8 +153,7 @@ function frameIdForPage(pageId: string): string {
149153

150154
/** One URL per image type, so the shared response's content type is right for every override of it. */
151155
export function screenshotUrl(screenshot: string): string {
152-
const extension = imageMime(screenshot).slice('image/'.length)
153-
return `${SCREENSHOT_URL_ORIGIN}/action.${extension}`
156+
return `${SCREENSHOT_URL_ORIGIN}/action.${imageExtension(screenshot)}`
154157
}
155158

156159
function imageDocument(snap: ActionSnapshot): FrameSnapshotNode {
@@ -197,7 +200,11 @@ export function buildImageFrameSnapshots(
197200
resourceOverrides: [
198201
{
199202
url: screenshotUrl(ref.snapshot.screenshot ?? ''),
200-
sha1: screenshotResourceName(pageId, ref.snapshot.timestamp)
203+
sha1: screenshotResourceName(
204+
pageId,
205+
ref.snapshot.timestamp,
206+
ref.snapshot.screenshot ?? ''
207+
)
201208
}
202209
],
203210
isMainFrame: true
@@ -232,7 +239,8 @@ export function buildScreenshotResourceSnapshots(
232239
})
233240
har.snapshot.response.content._sha1 = screenshotResourceName(
234241
pageId,
235-
snapshot.timestamp
242+
snapshot.timestamp,
243+
screenshot
236244
)
237245
byUrl.set(url, {
238246
type: 'resource-snapshot',

‎packages/trace/src/trace-snapshots.ts‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
// from them. Split from trace-exporter.ts; the exporter composes these into
33
// the trace event stream.
44

5-
import type { ActionSnapshot } from '@wdio/devtools-shared'
5+
import { imageExtension, type ActionSnapshot } from '@wdio/devtools-shared'
66
import type { TraceZipResource } from './trace-zip-writer.js'
77

88
export interface ScreencastFrameEvent {
@@ -70,12 +70,13 @@ export function upsertRichestSnapshot(
7070
mergeSnapshotInto(snapshots[idx]!, snap)
7171
}
7272

73-
/** Name under `resources/` of the screenshot captured at `timestamp`. */
73+
/** Name under `resources/` of a snapshot's screenshot, extension sniffed from its bytes. */
7474
export function screenshotResourceName(
7575
pageId: string,
76-
timestamp: number
76+
timestamp: number,
77+
screenshot: string
7778
): string {
78-
return `${pageId}-${timestamp}.jpeg`
79+
return `${pageId}-${timestamp}.${imageExtension(screenshot)}`
7980
}
8081

8182
export function buildSnapshotResources(
@@ -88,7 +89,11 @@ export function buildSnapshotResources(
8889
const base = `${pageId}-${snap.timestamp}`
8990
if (snap.screenshot) {
9091
out.push({
91-
resourceName: screenshotResourceName(pageId, snap.timestamp),
92+
resourceName: screenshotResourceName(
93+
pageId,
94+
snap.timestamp,
95+
snap.screenshot
96+
),
9297
data: Buffer.from(snap.screenshot, 'base64')
9398
})
9499
}
@@ -118,7 +123,7 @@ function frameForSnapshot(
118123
const frame: ScreencastFrameEvent = {
119124
type: 'screencast-frame',
120125
pageId,
121-
sha1: screenshotResourceName(pageId, snap.timestamp),
126+
sha1: screenshotResourceName(pageId, snap.timestamp, snap.screenshot ?? ''),
122127
width: viewport.width,
123128
height: viewport.height,
124129
timestamp

‎packages/trace/tests/screencast-trace.test.ts‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,28 @@ describe('buildDenseScreencast', () => {
126126
})
127127
})
128128

129+
it('names each frame resource by its own image format', () => {
130+
const png = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJ'
131+
const jpeg = '/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAMCAgICAgMC'
132+
const { events, resources } = buildDenseScreencast(
133+
[
134+
{ data: png, timestamp: 0 },
135+
{ data: jpeg, timestamp: 200 }
136+
],
137+
'page@x',
138+
0,
139+
viewport,
140+
{ minFrameIntervalMs: 100 }
141+
)
142+
expect(resources.map((r) => r.resourceName.split('.')[1])).toEqual([
143+
'png',
144+
'jpeg'
145+
])
146+
expect(events.map((e) => e.sha1)).toEqual(
147+
resources.map((r) => r.resourceName)
148+
)
149+
})
150+
129151
it('resource bytes decode back to the source frame data', () => {
130152
const { resources } = buildDenseScreencast(
131153
[frame('hello', 0)],

‎packages/trace/tests/trace-frame-snapshots.test.ts‎

Lines changed: 66 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
import { describe, it, expect } from 'vitest'
2-
import { strFromU8, unzipSync } from 'fflate'
2+
import { strFromU8, strToU8, unzipSync, zipSync } from 'fflate'
33
import {
44
buildActionEvents,
55
buildImageFrameSnapshots,
@@ -10,6 +10,7 @@ import {
1010
type AfterEvent,
1111
type FrameSnapshotEvent,
1212
type FrameSnapshotRef,
13+
type ScreencastFrameEvent,
1314
type ScreenshotResourceEvent
1415
} from '@wdio/devtools-trace'
1516
import { parseTraceZip } from '../src/trace-reader.js'
@@ -354,7 +355,7 @@ describe('buildScreenshotResourceSnapshots', () => {
354355
[
355356
'https://screenshot.wdio-devtools.invalid/action.png',
356357
'image/png',
357-
'page@abc123-2000.jpeg'
358+
'page@abc123-2000.png'
358359
],
359360
[
360361
'https://screenshot.wdio-devtools.invalid/action.jpeg',
@@ -461,13 +462,49 @@ describe('a written trace stores each screenshot once', () => {
461462
expect(
462463
frameSnapshots.map((e) => e.snapshot.resourceOverrides[0]!.sha1)
463464
).toEqual([
464-
'page@abc12345-1000.jpeg',
465+
'page@abc12345-1000.png',
465466
'page@abc12345-2000.jpeg',
466467
'page@abc12345-2000.jpeg'
467468
])
468469
expect(responseUrls).toEqual([screenshotUrl(PNG), screenshotUrl(JPEG)])
469470
})
470471

472+
it('names each screenshot by its own format in every place that names it', async () => {
473+
const { files, events } = await written()
474+
const stored = Object.keys(files)
475+
.filter((name) => /\.(png|jpeg)$/.test(name))
476+
.sort()
477+
expect(stored).toEqual([
478+
'resources/page@abc12345-1000.png',
479+
'resources/page@abc12345-2000.jpeg'
480+
])
481+
const filmstrip = (
482+
events.filter(
483+
(e) => e.type === 'screencast-frame'
484+
) as ScreencastFrameEvent[]
485+
).map((e) => e.sha1)
486+
const overrides = (
487+
events.filter((e) => e.type === 'frame-snapshot') as FrameSnapshotEvent[]
488+
).map((e) => e.snapshot.resourceOverrides[0]!.sha1)
489+
const responses = (
490+
events.filter(
491+
(e) => e.type === 'resource-snapshot'
492+
) as ScreenshotResourceEvent[]
493+
).map((e) => e.snapshot.response.content)
494+
for (const name of [
495+
...filmstrip,
496+
...overrides,
497+
...responses.map((c) => c._sha1)
498+
]) {
499+
expect(stored).toContain(`resources/${name}`)
500+
}
501+
expect(new Set(filmstrip)).toEqual(new Set(overrides))
502+
expect(responses.map((c) => [c.mimeType, c._sha1])).toEqual([
503+
['image/png', 'page@abc12345-1000.png'],
504+
['image/jpeg', 'page@abc12345-2000.jpeg']
505+
])
506+
})
507+
471508
it('reads back unchanged, with the screenshot responses kept out of the network rows', async () => {
472509
const { zip } = await written()
473510
const data = parseTraceZip(zip)
@@ -480,4 +517,30 @@ describe('a written trace stores each screenshot once', () => {
480517
expect(data.frames.map((f) => f.screenshot)).toEqual([PNG, JPEG])
481518
expect(data.trace.commands[1]!.snapshotText).toBe('- button "Go"')
482519
})
520+
521+
it('still reads a zip written when every screenshot was named .jpeg', async () => {
522+
const { files } = await written()
523+
const legacy: Record<string, Uint8Array> = {}
524+
for (const [name, data] of Object.entries(files)) {
525+
legacy[name.replace(/\.png$/, '.jpeg')] = data
526+
}
527+
legacy['trace.trace'] = strToU8(
528+
strFromU8(files['trace.trace']!).replaceAll(
529+
'page@abc12345-1000.png',
530+
'page@abc12345-1000.jpeg'
531+
)
532+
)
533+
expect(legacy['resources/page@abc12345-1000.jpeg']).toBeDefined()
534+
expect(Object.keys(legacy).some((name) => name.endsWith('.png'))).toBe(
535+
false
536+
)
537+
538+
const data = parseTraceZip(zipSync(legacy))
539+
expect(data.frames.map((f) => f.screenshot)).toEqual([PNG, JPEG])
540+
expect(data.trace.commands.map((c) => c.screenshot)).toEqual([
541+
PNG,
542+
JPEG,
543+
JPEG
544+
])
545+
})
483546
})

0 commit comments

Comments
 (0)