Skip to content

Commit e104207

Browse files
sfwclaude
andcommitted
verifier: Pareto admission gate (Phase 4 of self-evolution)
The journal-prescribed insight r-46988c97 (conf 0.78) requires "living archive admission gated by Pareto dominance across externally validated axes". Implement the multi-axis admission check and add register_admission_mode setting to opt in. Default axis set (4 axes, after distribution-analysis pre-flight): - verified_confidence - premises_supported_count - peer_differentiators_count - inverse_alias_gap known_prior_art_score is excluded by default — it's all-zero on journals without curated known_prior_art anchors (the common case), so it would be a Pareto no-op. Re-includable when a journal does maintain anchors AND verifier evaluations produce differentiator lists. The full 5-axis dict is still STORED on every entry so flipping the axis set later doesn't require reverification. How it works: - register_admission_mode = "scalar" (default — backward compat): existing single-floor admission unchanged. - register_admission_mode = "pareto": after the scalar gate approves, also run a Pareto-dominance check against existing active entries. Reject the candidate if any existing entry beats it on every axis (>= on all, strictly > on at least one). Catches "just like X but slightly worse on every dimension" admissions the scalar floor cannot see. Implementation: - _compute_pareto_axes: single source of truth for the 5-axis dict, used by verify_insight (new candidates) and the backfill pass (existing entries). Refactor of previously inlined code. - _pareto_dominates(existing, candidate): >= on all axes, > on at least one. Uses _PARETO_AXES tuple as the axis set. - _check_pareto_admission(candidate): O(N) scan of register; returns (admitted, dominating_entry_ids). - verify_insight: pareto_axes computed once before _register_gate; Pareto check runs after gate approves (only when admission_mode is pareto). Storage path unchanged (axes always persisted). Smoke test against the ideation_on_ideation register (21 entries): - existing Pareto frontier = 8 entries, each holding ≥1 axis - strictly-worse candidate → rejected (dominated by 5 entries) - identical-to-r-c6745772 candidate → admitted (Pareto allows ties) - mediocre-but-novel-on-alias_gap candidate → admitted via the structural-distinctness axis Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent fc1033a commit e104207

4 files changed

Lines changed: 170 additions & 28 deletions

File tree

config.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,21 @@ class EngineSettings:
149149
# confidence on a verdict change is a hedge pattern. This floor keeps
150150
# stored confidence honest. Set to 0.0 to disable.
151151
confidence_drop_on_downgrade: float = 0.10
152+
# Register admission policy. Two modes:
153+
# "scalar" (default — backward compatible): a candidate is admitted
154+
# iff verdict is validated AND verified_confidence >=
155+
# register_confidence_floor AND premises_supported AND novelty
156+
# isn't restatement/unsupported. Single-axis quality bar.
157+
# "pareto": ALL of the scalar mode's checks PLUS a Pareto-dominance
158+
# check against the existing register. A new candidate is rejected
159+
# if any existing active entry dominates it across the 4-axis
160+
# Pareto set: verified_confidence × premises_supported_count ×
161+
# peer_differentiators_count × inverse_alias_gap. Catches the
162+
# failure mode where a new entry is "just like X but slightly
163+
# worse on every axis" — an admission that the scalar floor
164+
# can't see.
165+
# Phase 4 of the self-evolving verifier.
166+
register_admission_mode: str = "scalar"
152167
# Questions below this priority are rejected at enqueue time (except
153168
# human-sourced questions, which always bypass). Default 0.0 = disabled.
154169
# An earlier default of 0.70 was found to starve new journals — early
@@ -312,6 +327,9 @@ def load(cls, path: Path = CONFIG_PATH) -> CuriosityEngineConfig:
312327
question_priority_floor=float(
313328
eng_section.get("question_priority_floor", 0.70)
314329
),
330+
register_admission_mode=str(
331+
eng_section.get("register_admission_mode", "scalar")
332+
).strip().lower() or "scalar",
315333
held_entries_enabled=bool(eng_section.get("held_entries_enabled", True)),
316334
held_confidence_floor=float(eng_section.get("held_confidence_floor", 0.7)),
317335
cross_ref_role=str(eng_section.get("cross_ref_role", "")).strip(),
@@ -671,6 +689,13 @@ def _build_toml(
671689
# before the journal could build context). Set to a non-zero value only on
672690
# mature journals where you specifically want to prune low-priority noise.
673691
question_priority_floor = {eng.question_priority_floor}
692+
# Register admission mode. "scalar" = single confidence floor + status checks
693+
# (default; backward compatible). "pareto" = ALSO require the new entry to be
694+
# non-dominated by any existing active entry across the 4-axis Pareto set
695+
# (verified_confidence × premises_supported_count × peer_differentiators_count
696+
# × inverse_alias_gap). Pareto rejects "just like X but slightly worse on
697+
# every axis" admissions that the scalar floor can't see.
698+
register_admission_mode = "{eng.register_admission_mode}"
674699
# Held-state pipeline — when the verifier returns `inconclusive` (couldn't reach
675700
# the claim, not refuted it), insights become held register entries pending
676701
# settlement rather than being silently rejected. Held entries usually require

engine/verification.py

Lines changed: 116 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -304,6 +304,82 @@ def _alias_gap(self, candidate: dict, *, exclude_id: str = "") -> dict:
304304
"scored_against": scored,
305305
}
306306

307+
# ── Pareto admission (Phase 4) ──────────────────────────────────────
308+
309+
# Default axis set for the Pareto admission gate. Excludes
310+
# `known_prior_art_score` because it's degenerate (all-zero) on
311+
# journals that have no human-curated known_prior_art anchors —
312+
# which is the common case. When a journal does maintain anchors
313+
# AND verifier evaluations produce differentiator lists, that
314+
# axis can be re-enabled via config (future work; not exposed
315+
# in Phase 4).
316+
_PARETO_AXES = (
317+
"verified_confidence",
318+
"premises_supported_count",
319+
"peer_differentiators_count",
320+
"inverse_alias_gap",
321+
)
322+
323+
@staticmethod
324+
def _compute_pareto_axes(
325+
*,
326+
verified_confidence: float,
327+
premises_support_citations: list,
328+
closest_peer_system: dict,
329+
known_prior_art_evaluations: list,
330+
alias_gap: float,
331+
) -> dict:
332+
"""Single source of truth for an entry's pareto_axes dict.
333+
Used by verify_insight (new candidates) and the backfill pass
334+
(existing entries). The full 5-axis dict is stored on the entry,
335+
even though the default Pareto check uses only 4 axes — keeps
336+
future axis tuning possible without re-running verification."""
337+
kpa = known_prior_art_evaluations or []
338+
return {
339+
"verified_confidence": float(verified_confidence),
340+
"premises_supported_count": len(premises_support_citations or []),
341+
"peer_differentiators_count": len(
342+
(closest_peer_system or {}).get("differentiators") or []
343+
),
344+
"known_prior_art_score": (
345+
float(sum(1 for ev in kpa if ev.get("differentiators")))
346+
/ max(1.0, len(kpa))
347+
),
348+
"inverse_alias_gap": float(alias_gap),
349+
}
350+
351+
@classmethod
352+
def _pareto_dominates(cls, existing_axes: dict, candidate_axes: dict) -> bool:
353+
"""True iff `existing` dominates `candidate` (>= on every axis,
354+
AND > on at least one). Missing axes are treated as 0.0."""
355+
if not existing_axes or not candidate_axes:
356+
return False
357+
strict_better = False
358+
for axis in cls._PARETO_AXES:
359+
e = float(existing_axes.get(axis, 0.0))
360+
c = float(candidate_axes.get(axis, 0.0))
361+
if e < c:
362+
return False # existing loses on this axis → cannot dominate
363+
if e > c:
364+
strict_better = True
365+
return strict_better
366+
367+
def _check_pareto_admission(self, candidate_axes: dict) -> tuple[bool, list[str]]:
368+
"""Returns (admitted, dominating_entry_ids). admitted=True iff no
369+
existing register entry dominates the candidate on the configured
370+
axis set. Entries without `pareto_axes` populated do not
371+
participate in the comparison (they predate Phase 1)."""
372+
dominating: list[str] = []
373+
for e in self.journal.register:
374+
if e.get("status") != "active":
375+
continue
376+
ex = e.get("pareto_axes") or {}
377+
if not ex:
378+
continue
379+
if self._pareto_dominates(ex, candidate_axes):
380+
dominating.append(e.get("id", ""))
381+
return (not dominating, dominating)
382+
307383
# ── Component-resolved novelty (Phase 3) ────────────────────────────
308384

309385
@staticmethod
@@ -936,11 +1012,43 @@ def verify_insight(
9361012
peer_has_differentiators = bool(
9371013
closest_peer_system.get("differentiators") or []
9381014
)
1015+
1016+
# Phase 4: compute pareto_axes once and persist on every entry
1017+
# (regardless of admission mode). Even in scalar mode the values
1018+
# are stored so that flipping admission_mode → pareto later
1019+
# doesn't require backfilling.
1020+
pareto_axes = self._compute_pareto_axes(
1021+
verified_confidence=verified_confidence,
1022+
premises_support_citations=result.get("premises_support_citations", []) or [],
1023+
closest_peer_system=closest_peer_system,
1024+
known_prior_art_evaluations=known_prior_art_evaluations,
1025+
alias_gap=alias_signal.get("gap", 1.0),
1026+
)
1027+
9391028
outcome, entry_status, gate_reasons = self._register_gate(
9401029
verdict, verified_confidence, premises_supported, synthesis_findable,
9411030
novelty_type=novelty_type,
9421031
peer_has_differentiators=peer_has_differentiators,
9431032
)
1033+
1034+
# Phase 4: Pareto admission check, applied AFTER the scalar gate
1035+
# approves. New entry is rejected if any existing active entry
1036+
# dominates it on the configured axis set. Scalar mode → no-op.
1037+
admission_mode = (
1038+
getattr(self.config, "register_admission_mode", "scalar") or "scalar"
1039+
).strip().lower()
1040+
if outcome == "register" and admission_mode == "pareto":
1041+
admitted, dominating = self._check_pareto_admission(pareto_axes)
1042+
if not admitted:
1043+
ids = ", ".join(dominating[:3])
1044+
more = f" (+{len(dominating) - 3} more)" if len(dominating) > 3 else ""
1045+
print(
1046+
f" [pareto admission] candidate dominated by {ids}{more} "
1047+
f"on axes {list(self._PARETO_AXES)} — rejecting."
1048+
)
1049+
outcome = "reject"
1050+
gate_reasons = list(gate_reasons) + [f"pareto_dominated_by={dominating[:3]}"]
1051+
9441052
if outcome == "reject":
9451053
print(f" Not registered ({', '.join(gate_reasons)}).")
9461054
return None
@@ -1009,20 +1117,7 @@ def verify_insight(
10091117
known_prior_art_evaluations=list(known_prior_art_evaluations),
10101118
canonical_form=dict(canonical_form),
10111119
component_novelty=dict(component_novelty),
1012-
pareto_axes={
1013-
"verified_confidence": verified_confidence,
1014-
"premises_supported_count": len(
1015-
result.get("premises_support_citations", []) or []
1016-
),
1017-
"peer_differentiators_count": len(
1018-
(closest_peer_system.get("differentiators") or [])
1019-
),
1020-
"known_prior_art_score": float(sum(
1021-
1 for ev in known_prior_art_evaluations
1022-
if ev.get("differentiators")
1023-
)) / max(1.0, len(known_prior_art_evaluations)),
1024-
"inverse_alias_gap": alias_signal.get("gap", 1.0),
1025-
},
1120+
pareto_axes=pareto_axes,
10261121
)
10271122

10281123
self.journal.add_register_entry(register_entry)
@@ -1891,20 +1986,13 @@ def backfill_canonical_forms(self, force: bool = False) -> dict:
18911986
)[:200]
18921987
)
18931988
if not entry.get("pareto_axes") or force:
1894-
peer = entry.get("closest_peer_system") or {}
1895-
kpa = entry.get("known_prior_art_evaluations") or []
1896-
entry["pareto_axes"] = {
1897-
"verified_confidence": float(entry.get("verified_confidence", 0.0)),
1898-
"premises_supported_count": len(
1899-
entry.get("premises_support_citations", []) or []
1900-
),
1901-
"peer_differentiators_count": len(peer.get("differentiators") or []),
1902-
"known_prior_art_score": (
1903-
float(sum(1 for ev in kpa if ev.get("differentiators")))
1904-
/ max(1.0, len(kpa))
1905-
),
1906-
"inverse_alias_gap": gap,
1907-
}
1989+
entry["pareto_axes"] = self._compute_pareto_axes(
1990+
verified_confidence=float(entry.get("verified_confidence", 0.0)),
1991+
premises_support_citations=entry.get("premises_support_citations", []) or [],
1992+
closest_peer_system=entry.get("closest_peer_system") or {},
1993+
known_prior_art_evaluations=entry.get("known_prior_art_evaluations") or [],
1994+
alias_gap=gap,
1995+
)
19081996
stats["canonicalized"] += 1
19091997

19101998
# Persist once at the end — the journal save is atomic so we don't

web/main.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1546,6 +1546,7 @@ def settings_save(
15461546
engine_gap_verification_hit_threshold: int = Form(5),
15471547
engine_confidence_drop_on_downgrade: float = Form(0.10),
15481548
engine_question_priority_floor: float = Form(0.70),
1549+
engine_register_admission_mode: str = Form("scalar"),
15491550
engine_analog_probe_max_analogs: int = Form(3),
15501551
engine_assumption_probe_max_assumptions: int = Form(3),
15511552
):
@@ -1605,6 +1606,10 @@ def _checkbox(v: str) -> bool:
16051606
if gap_scan_classify_role and gap_scan_classify_role not in known_roles:
16061607
gap_scan_classify_role = ""
16071608

1609+
register_admission_mode = engine_register_admission_mode.strip().lower()
1610+
if register_admission_mode not in ("scalar", "pareto"):
1611+
register_admission_mode = "scalar"
1612+
16081613
# Render any extra [models.<name>] profiles back out verbatim — we don't
16091614
# expose them in the web form yet, but we mustn't erase them on save.
16101615
extras_toml = ""
@@ -1667,6 +1672,7 @@ def _checkbox(v: str) -> bool:
16671672
f"gap_verification_hit_threshold = {max(1, min(100, engine_gap_verification_hit_threshold))}\n"
16681673
f"confidence_drop_on_downgrade = {max(0.0, min(0.5, engine_confidence_drop_on_downgrade))}\n"
16691674
f"question_priority_floor = {max(0.0, min(1.0, engine_question_priority_floor))}\n"
1675+
f'register_admission_mode = "{register_admission_mode}"\n'
16701676
f"analog_probe_max_analogs = {max(1, min(10, engine_analog_probe_max_analogs))}\n"
16711677
f"assumption_probe_max_assumptions = {max(1, min(10, engine_assumption_probe_max_assumptions))}\n"
16721678
)

web/templates/settings.html

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -529,6 +529,29 @@ <h3 class="text-xs uppercase tracking-wider text-slate-300 mb-1">Parallel fan-ou
529529
value="{{ connection.engine.question_priority_floor }}"
530530
class="mt-1 w-32 px-3 py-2 rounded-lg border border-slate-700 bg-slate-900/60 text-sm mono" />
531531
</label>
532+
533+
<label class="block md:col-span-3">
534+
<span class="text-xs uppercase tracking-wider text-slate-400">register_admission_mode</span>
535+
<span class="block text-[11px] text-slate-500">
536+
How the engine decides which validated insights enter the durable register.
537+
<span class="mono">scalar</span> (default — backward compatible): single confidence floor +
538+
status checks. <span class="mono">pareto</span>: ALSO require the new entry to be non-dominated
539+
by any existing active entry across the 4-axis Pareto set
540+
(<span class="mono">verified_confidence</span> × <span class="mono">premises_supported_count</span> ×
541+
<span class="mono">peer_differentiators_count</span> × <span class="mono">inverse_alias_gap</span>).
542+
Pareto rejects "just like X but slightly worse on every axis" admissions that the
543+
scalar floor can't see. Phase 4 of the self-evolving verifier.
544+
</span>
545+
<select name="engine_register_admission_mode"
546+
class="mt-1 w-full md:w-64 px-3 py-2 rounded-lg border border-slate-700 bg-slate-900/60 text-sm mono">
547+
<option value="scalar" {% if (connection.engine.register_admission_mode or 'scalar') == 'scalar' %}selected{% endif %}>
548+
scalar (single confidence floor)
549+
</option>
550+
<option value="pareto" {% if connection.engine.register_admission_mode == 'pareto' %}selected{% endif %}>
551+
pareto (4-axis non-dominated check)
552+
</option>
553+
</select>
554+
</label>
532555
</div>
533556
</section>
534557

0 commit comments

Comments
 (0)