Refine TwinBench v1 surface and archive legacy materials

Author: Muhra95

LEADERBOARD.md

@@ -1,41 +1,51 @@
 # TwinBench Leaderboard
 
-This file is the public template for reported TwinBench results. It is intentionally plain. Benchmark repositories should optimize for comparability, not presentation.
+TwinBench v1 is ready for external result submission. The canonical leaderboard structure is defined here even though no public external submissions are listed yet.
+
+## Current Status
+
+- Public external submissions: `none listed yet`
+- Canonical benchmark version: `1.0`
+- Example artifact: [results/reference-example-v1.json](results/reference-example-v1.json)
+
+The example artifact is a schema and reporting reference, not a competitive leaderboard entry.
 
 ## Reporting Rules
 
-- Benchmark version must be stated explicitly.
-- Scenario deviations must be disclosed in `Notes / Caveats`.
-- Partial runs may be listed, but incomplete coverage should be made visible.
-- Scores are reported on a `0-100` scale.
+- Every row must include the benchmark version used.
+- Scenario deviations must be disclosed.
+- Evidence references should be included whenever available.
+- Coverage shortfalls must remain visible.
+- Evaluator notes and caveats are part of the result, not optional extras.
 
 ## Leaderboard Template
 
-| System Name | Version | Date Evaluated | MR | IC | CCC | TC | PG | Total Score | Notes / Caveats |
-|-------------|---------|----------------|----|----|-----|----|----|-------------|-----------------|
-| Example System | 0.1.0 | 2026-04-03 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | Template row |
+| System Name | System Version | Evaluation Date | Benchmark Version | MR | IC | CCC | TC | PG | Total Score | Coverage | Evidence | Evaluator Notes | Caveats |
+|-------------|----------------|-----------------|-------------------|----|----|-----|----|----|-------------|----------|----------|-----------------|---------|
+| No public submissions yet | - | - | 1.0 | - | - | - | - | - | - | - | [Reference example](results/reference-example-v1.json) | Template row only | Not a leaderboard entry |
 
 ## Result Record Template
 
 ```json
 {
   "benchmark_name": "TwinBench",
+  "benchmark_title": "TwinBench: Benchmark for Persistent AI Systems",
   "benchmark_version": "1.0",
   "system_name": "Example System",
   "system_version": "0.1.0",
   "date_evaluated": "2026-04-03",
   "metrics": {
-    "MR": 0.0,
-    "IC": 0.0,
-    "CCC": 0.0,
-    "TC": 0.0,
-    "PG": 0.0
+    "MR": {"score": 0.0},
+    "IC": {"score": 0.0},
+    "CCC": {"score": 0.0},
+    "TC": {"score": 0.0},
+    "PG": {"score": 0.0}
   },
   "total_score": 0.0,
   "scenario_coverage": 0.0,
   "metric_coverage": 0.0,
-  "notes": [
-    "List scenario deviations, evaluator caveats, or partial coverage here."
-  ]
+  "evidence": [],
+  "evaluator_notes": [],
+  "caveats": []
 }
 ```

METRICS.md

@@ -1,26 +1,20 @@
-# TwinBench Metrics v1
+# TwinBench Metrics
 
-TwinBench v1 defines five core metrics. Each metric is scored on a `0-100` scale. The benchmark default is equal metric weighting.
+TwinBench v1 defines five core metrics. Each metric is scored on a `0-100` scale. The default weighting is equal across metrics.
 
 ## 1. Memory Retention (MR)
 
 **Definition**
 
-Memory Retention measures whether a system preserves and correctly recalls previously established user-relevant information across session boundaries and elapsed time.
+Memory Retention measures whether the system preserves and correctly recalls previously established user-relevant information across session boundaries and elapsed time.
 
 **What it measures**
 
-- persistence of facts, preferences, and prior decisions
-- resistance to forgetting after context resets
-- correct recall after delay or intervening interaction
+- durable recall of facts, preferences, and prior decisions
+- resistance to forgetting after delay or context reset
+- correct handling of updated or contradictory information
 
-**Why it matters**
-
-Persistent systems are not useful if continuity collapses at the end of a session. MR captures whether memory is durable enough to support longitudinal interaction.
-
-**Scoring method**
-
-MR is computed from recall probes across delayed checkpoints.
+**Scoring approach**
 
 ```text
 MR = 0.50 * factual_recall
@@ -30,137 +24,129 @@ MR = 0.50 * factual_recall
 
 Each component is normalized to `0-100`.
 
-**Example pass behavior**
+**Evidence requirements**
 
-- The system recalls a user’s standing project preference after several sessions.
-- The system updates an obsolete preference when corrected and uses the new value later.
+- at least one delayed recall probe
+- at least one memory update or contradiction case
+- evaluator notes when recall depends on leading hints
 
-**Example fail behavior**
+**Pass / fail intuition**
 
-- The system forgets the preference after a restart.
-- The system recalls the old preference after being explicitly updated.
+- Pass: the system recalls stable facts and applies corrected information later.
+- Fail: the system forgets the fact, reverts to stale information, or invents unsupported memory.
 
 **Caveats**
 
-- Retrieval success can reflect prompt phrasing as well as true memory quality.
-- Some systems summarize aggressively; evaluators should distinguish compression from failure.
+- prompt wording can affect recall quality
+- compressed summaries may obscure whether the underlying memory was preserved precisely
 
 ## 2. Identity Consistency (IC)
 
 **Definition**
 
-Identity Consistency measures whether the system maintains a stable representation of the user and of its own role, commitments, and conversational posture over time.
+Identity Consistency measures whether the system maintains a stable representation of the user and a stable understanding of its own role, commitments, and collaboration norms over time.
 
 **What it measures**
 
-- user identity continuity
-- stable self-description and role adherence
-- consistency of standing commitments, plans, and constraints
+- continuity of the user profile
+- stability of the system’s declared role
+- consistency of standing commitments and working norms
 
-**Why it matters**
-
-A persistent system should not behave as though the user is new in every session or as though its own operating role changes arbitrarily over time.
-
-**Scoring method**
+**Scoring approach**
 
 ```text
 IC = 0.40 * user_identity_stability
    + 0.35 * system_role_stability
    + 0.25 * commitment_consistency
 ```
 
-**Example pass behavior**
+**Evidence requirements**
 
-- The system continues addressing the same user profile and respects previously established working norms.
-- It retains its declared role as a planning assistant rather than drifting into unrelated personas.
+- at least two dated checkpoints
+- at least one probe of user understanding
+- at least one probe of system role or standing commitment
 
-**Example fail behavior**
+**Pass / fail intuition**
 
-- It confuses the user with another profile.
-- It contradicts its own prior commitments without a triggering event.
+- Pass: the system treats the same user as the same user and preserves the collaboration frame.
+- Fail: the system behaves as if the user is new, confuses identities, or drifts into an incompatible role without cause.
 
 **Caveats**
 
-- Stylistic variation is not necessarily identity drift.
-- Systems with adaptive tone may appear less consistent unless prompts are controlled carefully.
+- stylistic variation alone is not identity drift
+- highly adaptive systems require careful prompt control to avoid false positives
 
 ## 3. Cross-Context Coherence (CCC)
 
 **Definition**
 
-Cross-Context Coherence measures whether information, plans, and interpretations remain consistent when interaction moves across sessions, channels, or task contexts.
+Cross-Context Coherence measures whether information, plans, and interpretations remain aligned when interaction moves across sessions, channels, or task contexts.
 
 **What it measures**
 
 - transfer of relevant context between surfaces
-- coherent interpretation when tasks resume elsewhere
-- avoidance of context fragmentation
-
-**Why it matters**
+- consistency of plan state and summaries
+- avoidance of fragmentation across contexts
 
-Persistent systems are often used across web, chat, email, documents, and scheduled workflows. Coherence across those contexts is central to real-world usefulness.
-
-**Scoring method**
+**Scoring approach**
 
 ```text
 CCC = 0.45 * context_transfer_accuracy
     + 0.35 * plan_coherence
     + 0.20 * state_alignment
 ```
 
-**Example pass behavior**
+**Evidence requirements**
 
-- A task started in chat can be resumed in email without re-explaining its goal.
-- The system carries forward the same status summary across surfaces.
+- at least one explicit context switch
+- one resumed task or memory probe in the new context
+- recorded notes about whether transfer was native or evaluator-mediated
 
-**Example fail behavior**
+**Pass / fail intuition**
 
-- The system treats the resumed task as unrelated.
-- Different contexts produce incompatible versions of the same plan.
+- Pass: work started in one context resumes coherently in another.
+- Fail: the system produces incompatible summaries, stale state, or fragmented plans.
 
 **Caveats**
 
-- True multi-channel measurement may require bespoke instrumentation.
-- Human evaluators may need to judge when omitted context is harmless versus coherence failure.
+- true multi-surface evaluation may require instrumentation not available in all systems
+- some context transfer failures are ambiguous and need evaluator judgment
 
 ## 4. Task Continuity (TC)
 
 **Definition**
 
-Task Continuity measures whether a system can maintain progress on multi-step work over time, including after interruptions, delays, or context changes.
+Task Continuity measures whether the system preserves progress on multi-step work across interruptions, delays, and resumed interaction.
 
 **What it measures**
 
 - preservation of task state
 - correct resumption after interruption
-- continuity of dependencies, decisions, and next actions
-
-**Why it matters**
-
-Persistent intelligence is valuable when work survives beyond the immediate turn. TC captures whether a system can behave like a long-horizon collaborator rather than a short-lived executor.
+- continuity of milestones, dependencies, and next actions
 
-**Scoring method**
+**Scoring approach**
 
 ```text
 TC = 0.40 * stateful_resumption
    + 0.35 * milestone_progression
    + 0.25 * interruption_recovery
 ```
 
-**Example pass behavior**
+**Evidence requirements**
 
-- The system resumes a deferred research task from the last confirmed milestone.
-- It knows which subtasks were already completed and which remain open.
+- at least one multi-step task
+- at least one interruption or delay
+- evidence that prior milestones were preserved or lost
 
-**Example fail behavior**
+**Pass / fail intuition**
 
-- It restarts the task from scratch after every interruption.
-- It claims completion without preserving prior outputs or decisions.
+- Pass: the system resumes from the correct milestone and advances the task without repeating completed work.
+- Fail: the system restarts from scratch, duplicates work, or claims progress without preserved state.
 
 **Caveats**
 
-- TC depends on scenario design. Trivial tasks can overstate continuity quality.
-- Some systems may perform well only when state is exposed explicitly by the interface.
+- trivial tasks can make continuity look better than it is
+- some interfaces expose task state explicitly while others hide it, which affects evaluation difficulty
 
 ## 5. Personalization Gain (PG)
 
@@ -170,33 +156,30 @@ Personalization Gain measures whether the system becomes more useful after learn
 
 **What it measures**
 
-- utility improvement from remembered preferences
+- later application of learned preferences
 - reduction in repeated correction
-- adaptation to user-specific defaults and working style
-
-**Why it matters**
-
-Persistent systems should improve with use. If stored user knowledge does not yield better behavior, persistence has limited practical value.
+- user-specific efficiency improvement
 
-**Scoring method**
+**Scoring approach**
 
 ```text
 PG = 0.50 * preference_application
    + 0.30 * correction_reduction
    + 0.20 * user_specific_efficiency
 ```
 
-**Example pass behavior**
+**Evidence requirements**
 
-- After learning formatting and scheduling preferences, the system applies them without being reminded.
-- Output quality improves measurably in later interactions.
+- baseline behavior before preference learning
+- one or more explicit preference statements or corrections
+- a later comparable task where preference use can be observed
 
-**Example fail behavior**
+**Pass / fail intuition**
 
-- The system stores preferences but does not use them.
-- The user must repeat the same corrections in each session.
+- Pass: later outputs reflect learned preferences without restatement and require fewer corrections.
+- Fail: stored preferences are not used, are inconsistently used, or must be repeated each time.
 
 **Caveats**
 
-- PG is sensitive to task selection and evaluator baselines.
-- Apparent improvement can come from easier later prompts rather than genuine personalization unless scenarios are controlled carefully.
+- later tasks must be comparable enough to support a fair before-and-after judgment
+- apparent gain may reflect easier prompts rather than genuine personalization