Skip to content

Commit 95b1113

Browse files
committed
Record closed-loop failure of the fixture-winning acceptance policy
The correction_conditioned policy (6/6 on single-row fixtures) was implemented as an opt-in degraded_tracking_acceptance gate and A/B-replayed on Koide outdoor_hard_01a 180 s: baseline 0.582 m translation RMSE vs 62.7 m with the gate enabled. In closed loop the correction-vs-prediction cross-check is self-referential (each accepted degraded measurement becomes the next prediction) and the rejection- counted streak budget resets on every acceptance. The runtime change is reverted; next iteration needs sequence fixtures and a non-resetting degraded-accept budget. Replay validation stays mandatory before any runtime gate change.
1 parent fed3a7d commit 95b1113

1 file changed

Lines changed: 27 additions & 0 deletions

File tree

docs/development_plan.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,33 @@ started the same day:
253253
consider the `measurement_gate_policy` C++ change behind the public regression
254254
gates
255255

256+
#### 2026-06-12 closed-loop replay result: fixture winner fails end-to-end
257+
258+
The `correction_conditioned` policy was implemented as an opt-in
259+
`degraded_tracking_acceptance` gate (unit-tested against the same fixture
260+
values) and A/B-replayed on Koide `outdoor_hard_01a` `180 s` on an idle
261+
machine:
262+
263+
- baseline (scalar gate): translation RMSE `0.582 m`, ok rows `329/539`
264+
- degraded acceptance: translation RMSE `62.7 m`, ok rows `152/535`
265+
266+
Root cause: in closed loop the correction-vs-prediction cross-check is
267+
self-referential. Each accepted degraded measurement becomes the next
268+
prediction, so a drifting pose keeps producing small corrections and keeps
269+
"confirming" itself; and because the streak budget counts *rejections*, every
270+
acceptance resets it, making the budget unbounded in degraded regions. Single-
271+
step fixtures cannot expose either effect.
272+
273+
Consequences (the runtime change was reverted; the experiment stays):
274+
275+
1. the `measurement_acceptance` experiment needs sequence fixtures that replay
276+
a window of consecutive samples with closed-loop feedback, not single rows
277+
2. the next policy iteration must bound *consecutive degraded accepts* with a
278+
counter that only a genuinely below-threshold measurement resets, and/or
279+
bound the cumulative correction absorbed during a degraded phase
280+
3. fixture-level benchmark scores are necessary but not sufficient; replay
281+
validation stays mandatory before any runtime gate change
282+
256283
### 2026-06-11: Global localization G1 start
257284

258285
- `MAP_GRID` baseline merged: map-wide seed candidates (occupied-cell centroids with a

0 commit comments

Comments
 (0)