fix: content-anchored fixture matching + relaxed-turnIndex divergence warn/opt-out

[jpr5]

TL;DR — what this is

What aimock does: records real LLM API calls once, then replays the recorded responses in tests — no live API, deterministic, fast.

The change in one sentence: matching a replay to a recorded fixture is now anchored on the content of the request (the messages/tools being sent), not on the turn number it happened to be recorded at.

Why: the old logic treated "this is turn #3" as a hard requirement. If your test's conversation drifted by even one turn from when it was recorded, replay would fail to match and error out — brittle. Real conversations reorder, retry, and branch. Content is the stable identity; turn position is not.

What actually changed

• Replay: turnIndex is no longer a hard reject gate. A fixture matches if its content matches; turn position becomes a tiebreaker, not a veto.
• Record: unchanged — still strict (records exactly what happened, in order).
• Visibility: when a fixture matches at a different position than recorded, aimock emits a divergence warning (with a matchedBy reason), so drift is never silent.
• Escape hatch: AIMOCK_STRICT_TURN_INDEX=1 restores the old strict behavior.

Risk surfaces (public pkg, ~500k downloads/wk) → mitigations

Risk	What could go wrong	Mitigation
Looser matching → wrong fixture	Replay silently returns a fixture that isn't the right one	Content predicates are primary — a match still requires the request content to match. Position is only relaxed after content already matches. A truly wrong match needs two content-identical requests at different turns (rare outside showcase-style harnesses).
Silent false-green in tests	A test passes against a mismatched fixture, hiding a real regression	The divergence warn makes every position-drift visible; matchedBy / describeMatch explain why a fixture matched. We never hide the drift — we surface it.
Breaking change on a minor bump	Existing users depending on strict turn-index matching get different behavior	Change is additive (relaxation only as a fallback); the canonical case (record→replay at the same position) is byte-for-byte unchanged. AIMOCK_STRICT_TURN_INDEX=1 opt-out preserves old behavior → ships as minor, not major.
Surprise / hard to debug	"Why did this fixture match?"	New matchedBy field + a throttled, human-readable warning explain the match decision.

Bottom line: it trades a brittle, position-coupled gate for a content-anchored match that mirrors how conversations really behave — while keeping strict mode one env var away, recording untouched, and every divergence loud rather than silent. The empirical blast-radius check on our corpus showed zero wrong-fixture matches.

---

Summary

Makes aimock's fixture replay matching content-anchored: a fixture's recorded content/shape is the primary match key, and turnIndex is no longer a hard reject gate. This kills a class of false-misses where a multi-bubble / multi-turn run that drifts past its scripted turnIndex was spuriously rejected (empty assistant response), even though the content matched. The record path is unchanged — strict per-turn capture is preserved via MatchOptions.strictTurnIndex.

⚠️ Public replay-semantics change — please confirm intended

A fixture that previously force-missed on turnIndex will now match if its content predicates match. Empirically measured by replaying the real fixture corpora (showcase d6 + aimock examples) through the old vs new matcher — 786 fixture sets / 9769 requests:

• 3213 MISS→MATCH (the intended false-miss fixes), 0 MATCH→MISS, 0 wrong-fixture selections (the 69 MATCH→DIFF are all same-content twins / forward-progress).
• 0 changes at the canonical position (assistantCount == turnIndex) — divergence happens only when a conversation runs off-by-N from its scripted turn.

So this is a strict improvement on real corpora; it's a minor bump. (Content gating stays strict; multiple content matches → closest scripted turnIndex; all-ahead → unpositioned fallback.)

Not surprising anyone (detection + reversible opt-out)

For the divergence to be discoverable + reversible by the ~500k-downloads/wk user base:

• AIMOCK_STRICT_TURN_INDEX=1 — process opt-out that restores the legacy strict-turnIndex hard gate for replay (default = new content-anchored behavior). This reversible escape hatch is what keeps the change a minor bump.
• Surgical divergence warn — a once-per-fixture, logger-gated warn fires only when a served fixture's turnIndex was relaxed (i.e. the strict gate would have rejected it). Silent in programmatic use (logger default), so it never spams a test suite; for the typical record→replay user (who sits at the canonical position) divergence is rare, so the warn is a genuine, useful signal.
• Diagnostic fields turnIndexRelaxed / matchedBy on the match diagnostic for inspection.

Review

Converged via two full cr-loop passes (content-anchored matching, then the warn package) — 7 unbiased agents/round + confirmation rounds + Procedure-3 audits (0 promotions). Notable fixes caught in review: WeakSet-identity throttle (predicate/RegExp fixtures no longer collide), accurate matchedBy attribution, and the divergence-direction wording.

Test plan

• pnpm test — 4041 passed / 0 failed; tsc --noEmit clean; lint 0/0; build ok
• Confirm the intended replay semantics (above) before merge

Known follow-ups (pre-existing, not this PR)

• getSystemText/getTextContent present-but-empty "" stray-newline edge under useExactMatch.
• Handler input-validation: unguarded parts/content derefs (gemini/cohere/responses/bedrock/bedrock-converse) → TypeError on malformed JSON.

Not auto-merging — flagging the replay-semantics change for review.

[pkg-pr-new]

Open in StackBlitz

npm i https://pkg.pr.new/@copilotkit/aimock@276

commit: 1bd3cde