Skip to content

Latest commit

 

History

History
189 lines (122 loc) · 7.91 KB

File metadata and controls

189 lines (122 loc) · 7.91 KB

Paper-to-Repo Mapping

1. Main Question In Plain Language

The paper asks a narrow question: under one fixed total priced inference budget and a hard per-node local context ceiling, when does splitting inference across multiple workers actually beat keeping the work inside one strong workspace, and when do apparent gains disappear once budget fairness, topology, and verification are accounted for?

2. Three-Layer Objective Structure

Tractable task-level surrogate utility q_x(A)

This is the paper's explanatory layer. It is used to reason about exploration, redundancy, specialization, communication fidelity, diversity, dependence, coordination cost, consensus cost, latency, and priced additive expenditure. It is not the deployability objective.

Repo mapping:

  • Synthetic pilots compute q_x(A) explicitly from retained coverage, diversity, dependence, fidelity, coordination burdens, latency, and priced budget.
  • Summary tables report q_x(A) separately from deployable utility.
  • Any decomposition result is tagged tractable_surrogate.

Deployable task utility u_x^{dep}(A)

This is the end-to-end objective:

u_x^{dep}(A) = V_x(A) * kappa_x(A) * nu_x(A) * (1 - alpha_x(A)) - lambda * E[phi(B_A^{add})]

Repo mapping:

  • Run logs separately record value_proxy, coverage, conditional_selection_accuracy, hijack_risk, and priced cost.
  • Reporting distinguishes oracle versus proxy coverage.
  • Deployability claims in pilots are framed as pilot evidence, not theorem validation.

Structural layer for topology and interaction preservation

This layer studies when topology can or cannot preserve task-critical higher-order interactions. It is not reducible to a generic "teamwork helps" score.

Repo mapping:

  • Synthetic interaction tasks encode critical interactions and topology-sensitive attenuation.
  • Message-only, shared-memory, and lossy-relay classes are compared under the same budget.
  • Structural diagnostics are reported separately from surrogate or deployability outcomes.

3. Fixed-Budget Resource Model

The paper defines:

  • R_add = (C, M, L, V, H)
  • W_max as a separate nonadditive per-node local context ceiling

Interpretation:

  • C: worker or tool compute
  • M: explicit external memory operations
  • L: communication
  • V: verification
  • H: horizon or sequential depth

Repo mapping:

  • src/budget/accounting.py implements explicit charging for all five additive components.
  • W_max is enforced independently from priced spend.
  • Orchestration, routing, summarization, memory bookkeeping, and final verification are charged explicitly.
  • No communication, verification, or memory operation is free.

4. Architecture Classes From The Paper

The repository keeps the paper's exact class names and distinctions:

  • single-workspace
  • message-only
  • bounded-local-state
  • shared-memory
  • lossy summarize-relay

Repo mapping:

  • SingleWorkspaceArchitecture
  • MessageOnlyArchitecture
  • SharedMemoryArchitecture
  • LossyRelayArchitecture
  • bounded_local_state is represented as an explicit flag and enforced by the same W_max accounting

5. Fairness Principle For Baseline Comparison

The paper's fairness rule is not optional. Comparisons must match:

  • same priced budget
  • same tool access
  • same note privilege or scratchpad privilege
  • same deadline or latency constraint when applicable

Repo mapping:

  • Experiment configs declare one shared fairness block across compared architectures.
  • The runner validates common budget, W_max, tool access, note privilege, and deadline fields.
  • The strong single-workspace baseline is allowed to use retries, self-consistency, and self-critique within the same priced budget.

6. Key Measurable Or Proxy-Certifiable Quantities

Decomposability via Möbius interaction spectrum / truncation certificate

  • Paper status: proxy-certifiable in general.
  • Repo status: exact-oracle on synthetic tasks because the generator exposes the interaction structure; scaffolded or latent on benchmark adapters.

Signed partition-induced diversity

  • Implemented from evidence novelty, route specialization, and semantic dispersion relative to a matched single-workspace control.
  • Measurement-pipeline dependence is explicit in config and logs.

Signed shared-failure dependence rho(A,x)

  • Implemented as a simple observable approximation from within-run worker failures and overlap structure.
  • Labeled honestly as an approximation in docs and summaries.

Decision-preserving fidelity Gamma(A,x)

  • Implemented as a topology-sensitive proxy based on preserved critical modules and message-path attenuation.
  • Logged as proxy-certifiable rather than exact.

Coverage kappa_x(A)

  • Synthetic tasks report oracle coverage.
  • Benchmark-style adapters can report oracle coverage when exact candidate correctness is available; otherwise the harness uses explicit proxy labeling.

Conditional selection accuracy nu_x(A)

  • Observable in synthetic pilots because candidate correctness is known.

Post-selection hijack risk alpha_x(A)

  • Observable in synthetic pilots when a correct candidate exists but a wrong candidate overrides it.
  • Benchmark-style reporting keeps the proxy/oracle distinction explicit.

Latency penalty Lambda_x(A)

  • Implemented as an observable penalty from critical-path depth and deadline miss.

7. Scope Tags In The Paper

Structural

  • Architecture-class statements under explicit information-flow constraints.
  • In the repo: topology-sensitive diagnostics, critical interaction preservation, and fair baseline validation.

Tractable-surrogate

  • Statements proved inside the compositional-coverage model.
  • In the repo: q_x(A), exact cost-aware telescoping decomposition, synthetic coverage-margin comparisons.

Proxy-certified

  • Statements conditional on an operational proxy or calibration assumption.
  • In the repo: Gamma(A,x), attenuation-style topology diagnostics, truncation certificates on non-synthetic tasks.

Design rules / diagnostics rather than universal theorems

  • Expert-preserving aggregation rules, local phase-boundary diagnostics, and deployment-threshold decision rules.
  • In the repo: plotted diagnostics and pilot analysis helpers, not theorem claims.

8. What Can Be Implemented Now, What Needs Approximation, What Remains Conceptual

Implemented directly now

  • Fixed-budget accounting over C, M, L, V, H
  • Explicit W_max enforcement
  • Local-only Ollama adapter with default model gemma3:1b
  • Strong single-workspace matched baseline
  • Message-only, shared-memory, and lossy-relay experiment classes
  • Synthetic interaction tasks with exact latent structure
  • Structured JSONL logging
  • Oracle coverage, conditional selection accuracy, hijack risk, latency, and budget breakdown on synthetic tasks
  • Small benchmark-style adapters and sample assets for cheap pilot runs
  • Live prompt-and-answer task packs for local Ollama smoke, synthetic, verification, and topology pilots

Implemented with approximation

  • Gamma(A,x) as a declared topology-sensitive proxy
  • rho(A,x) as a simple observable approximation
  • Diversity metrics via declared evidence and claim clustering pipelines
  • Benchmark-style kappa_x(A) when only proxy screens are available
  • Local Ollama integration is implemented, but default public pilots stay symbolic for CPU-feasible reproducibility
  • Local phase-boundary diagnostics from finite pilot sweeps rather than asymptotic theory

Still conceptual or latent relative to the full theory

  • Full certified attenuation models for real benchmark tasks
  • Real-world recoverable critical interaction mass outside synthetic generators
  • Strong identifiability guarantees for latent structural quantities on open-ended tasks
  • Universal theorem-level transfer from pilot experiments to broad model families

The repository therefore implements a faithful experiment harness for the paper's distinctions, but not a claim that the paper has been empirically proved in full generality.