Skip to content

Commit 04beaa0

Browse files
authored
Merge pull request #483 from webdriverio/feat/479-conformance-timings-targets
Conformance rules for navigation timings and page-level assertion targets
2 parents d86038c + f359b1e commit 04beaa0

22 files changed

Lines changed: 665 additions & 136 deletions

‎CLAUDE.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -375,6 +375,7 @@ Documented divergences from the conventions above. They exist today as debt to b
375375
- **Two mechanisms resolve a command's target selector, and the WDIO one is wrong for interleaved handles.** Selenium keys a `WeakMap` on handle identity (`WebElement.id_` is a promise, so no id is readable when a command is invoked); the service uses a mutable last-selector in `service/src/command-selectors.ts`, which stamps the wrong locator for `const a = await $('#a'); const b = await $('#b'); await a.click()` (the Selenium side is covered in `selenium-devtools/tests/element-locators.test.ts`; the WDIO failure itself has **no** test — it is an unverified reading of `command-selectors.ts`). They cannot share a registry — WDIO's hook sees a *serialized* handle carrying an id string and never the live object, Selenium has the object and no readable id. Unifying would need the policy parameterized rather than the storage shared; until then the WDIO path is knowingly wrong in that case. Nightwatch is a third mechanism: it reads arg 0 through a per-kind allowlist (`assertTarget.ts`) rather than tracking handles at all, because its classic API takes selector strings. The allowlists are deliberately **not** derived from shared's `ACTION_MAP` — that table says how a command *renders* (its `Element` entries include WDIO commands called on a handle, with no selector argument), not whether arg 0 is an element definition.
376376
- Service renders expect-webdriverio matchers as single `expect.<matcher>` rows by **folding**, not stack/depth suppression (the old `#assertionDepth`/`#matcherStarted`/self-heal machinery is gone). The matcher's value-read (`toHaveText`→`getText`, `toExist`→`isExisting`, …) is captured as a normal command; `afterAssertion` then coalesces the synthesized `expect.*` row into that read in place — inheriting its callSource, screenshot, and timeline position — and the fold replaces **by timestamp, never a public `id`**: `id` is the per-worker `commandCounter`, which resets per spec, so stamping one lets the app's id-first `replaceCommand` swap a same-id row from another spec (duplicate rows + a fold from another spec vanishing, in multi-spec live mode). `beforeAssertion` arms the pending matcher (depth-counted so aliases like `toBeChecked`→`toBeSelected` fold once); a matcher that **hard-throws** — element never resolves, so expect-webdriverio's `waitUntil` rethrows and `afterAssertion` never fires — is synthesized at `afterTest`/`afterStep` from the throwing read, so a failing assertion renders as `expect.<matcher>` whether or not the element existed. Two limits: its error is then the read's (`Can't call getText on … element wasn't found`), not an assertion-phrased message; and `MATCHER_READ_COMMANDS` is a hand-maintained allowlist, so a matcher whose read isn't listed leaves its raw read visible alongside the `expect.*` row. Plain-value jest matchers (`expect(x).toBe(y)`) don't fire the ewdio hooks, so they aren't captured as rows.
377377
- **What the conformance scenario shows the adapters doing differently.** Each entry is a `KNOWN_GAPS` reason in `packages/conformance/src/allowlist.ts`, scoped to the differences it explains (one hunk, one moved step, or one test's reported state), and reported stale once they stop occurring, except an entry marked `timingDependent`, which is never reported stale and has to be removed by hand.
378+
- **The WDIO service boxes the last element it resolved for an assertion on the page title** (#481). `SessionCapturer.captureAssertCommand` stamps a `node:assert` row with `#lastResolvedSelector` and the service never feeds core's value registry, so `assert.notEqual(await browser.getTitle(), 'Secure Area')` after `$('h2').getText()` carries `h2`, where the other three adapters carry no target. The user's own `getTitle` has no row either: it is in `INTERNAL_COMMANDS`, because the service calls it for its own snapshot probes.
378379
- **A Selenium or Python wait has no row when it succeeds, and a condition row when it fails** (#464). A successful `driver.wait(until.elementLocated(...))` or `WebDriverWait(...).until(EC.presence_of_element_located(...))` is a folded `findElements`/`findElement` row, which ACTION_MAP leaves unmapped; a timed-out one is a `wait` row named after the condition and mapped to `Browser.waitForFunction` without a target, where the WDIO service and Nightwatch show `Element.waitForExist` on the selector. Python reaches the Node.js adapter's shape (`selenium-devtools/src/wait-tracking.ts`) from `selenium_devtools/wait_tracking.py`, which patches `WebDriverWait.until`/`until_not`, because the wait is a client-side loop the `execute` chokepoint sees only as its polls; an `expected_conditions` closure is named after its factory and captured arguments, since its own name is `_predicate`.
379380
- **A condition wait shows its reads in some adapters and one wait row in others** (#464). A passing wait on a condition that reads the page (`waitUntil(() => getText ...)`, `driver.wait(async () => ...)`, `WebDriverWait(...).until(lambda d: ...)`) is one `Browser.waitForFunction` row in the WebdriverIO service, which logs only the command on top of its stack and so hides the reads inside it; its reads and no wait row in Selenium and Python; and both in Nightwatch. The reads are what show a fold bug (one read folded over another), so the scenario keeps them visible where the adapter shows them.
380381

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ Want a portable artifact instead of a live UI (CI / agent diffing)? Switch to **
5252
| Browser console logs | ✅ | ✅ | ✅ | ✅ |
5353
| Source tab opens the test line that ran the command | ✅ tested | ✅ tested | ✅ tested | ✅ tested |
5454
| Metadata tab (session, capabilities, device) | ✅ | ✅ | ✅ | ✅ |
55-
| Performance timings on navigation rows | ✅ | ✅ | ✅ | ✅ |
55+
| Performance timings on navigation rows | ✅ tested | ✅ tested | ✅ tested | ✅ tested |
5656
| Failed command in the Errors tab, with the browser's message | ✅ tested | ✅ tested | ✅ tested | ✅ tested |
5757
| Run, Rerun and Run all from the dashboard | ⚠️ no Cucumber step rerun ¹ | ⚠️ no Cucumber step rerun ¹ | ⚠️ no Cucumber step rerun ¹ | ✅ |
5858
| Preserve & Rerun (compare a failed run with its rerun) | ✅ | ✅ | ✅ | ✅ |

0 commit comments

Comments
 (0)