Add native harness full benchmark summary

Author: wxj630

README.md

@@ -76,7 +76,13 @@ def _post_json(self, url: str, payload: Mapping[str, Any]) -> Dict[str, Any]:
                 with urllib.request.urlopen(request, timeout=int(self.timeout_seconds), context=context) as response:
                     raw = response.read().decode("utf-8", errors="replace")
                 return json.loads(raw)
-            except (urllib.error.URLError, TimeoutError, http.client.RemoteDisconnected, json.JSONDecodeError) as exc:
+            except (
+                urllib.error.URLError,
+                TimeoutError,
+                http.client.RemoteDisconnected,
+                http.client.IncompleteRead,
+                json.JSONDecodeError,
+            ) as exc:
                 last_error = exc
                 if attempt >= int(self.max_retries or 0):
                     break

README_ZH.md

@@ -1,13 +1,13 @@
 # Adapters
 
-Bayesian-Agent is designed to integrate with external agent harnesses without copying their code. This is one of the main reasons the project is not just another agent framework: the Bayesian layer can improve whichever harness emits verified trajectories.
+Bayesian-Agent now has a first-party native harness, and it still integrates with external agent harnesses without copying their code. This is one of the main reasons the project is not just another monolithic framework: the Bayesian layer can improve whichever harness emits verified trajectories.
 
 ## Adaptation Advantage
 
 Bayesian-Agent separates Skill evolution from task execution:
 
 ```text
-Harness executes -> Bayesian-Agent learns -> Adapter injects model-facing Skill/SOP text -> Harness reruns
+Native or external harness executes -> Bayesian-Agent learns -> Skill/SOP text updates -> Harness reruns
 ```
 
 That separation enables three deployment styles:
@@ -16,6 +16,28 @@ That separation enables three deployment styles:
 - repair only the failed tasks from an existing agent run
 - reuse the same Skill belief registry across compatible harnesses
 
+## Native Harness First
+
+The default execution backend is now the Bayesian-Agent native harness:
+
+```bash
+python experiments/run_benchmarks.py \
+  --harness bayesian-agent \
+  --model deepseek-v4-flash \
+  --bench core \
+  --mode all \
+  --limit 1
+```
+
+The native harness owns only the minimal execution substrate:
+
+- LLM: a small OpenAI-compatible chat client.
+- Tools: workspace-scoped `file_read`, `file_write`, `code_run`, and `finish`.
+- Memory: three layers, `hippocampus`, intermediate `state`, and persistent `cortex`.
+- Loop: turn execution, tool dispatch, transcript capture, usage accounting, and trajectory persistence.
+
+The harness layer is intentionally simple and efficient. Most capability improvement is meant to come from Bayesian Skill/SOP evolution, where verified trajectories update reusable procedures instead of hiding behavior inside a large runtime.
+
 ## Adapter Contract
 
 An external harness should satisfy the `AgentAdapter` protocol:
@@ -71,17 +93,25 @@ External systems should emit:
 
 Bayesian-Agent can then update beliefs, keep posterior audit artifacts, and render the next model-facing Skill/SOP text.
 
-## Planned Bayesian-Agent Harness
+## Optional Compatibility Backends
+
+External harnesses remain useful for comparison and transfer. Current optional backend names are:
+
+```bash
+--harness genericagent
+--harness mini-swe-agent
+--harness claude-code
+```
 
-Current experiments use GenericAgent as the backend harness. A dedicated Bayesian-Agent harness is planned so users can run the full loop without depending on GenericAgent, while still keeping GenericAgent and other frameworks as optional backends.
+Each backend should emit enough trajectory evidence for Bayesian-Agent to update Skill beliefs: task identity, outcome, failure mode, token usage, tool/runtime metadata, and artifacts.
 
 ## MinimalAgent Status
 
-MinimalAgent adapter support is intentionally not included in v0.4.
+MinimalAgent adapter support is intentionally not included in v0.5.
 
 The recommended path is:
 
-1. stabilize the GenericAgent boundary
+1. keep the native harness small and inspectable
 2. keep the core trace schema portable
-3. upload the dedicated Bayesian-Agent harness
+3. use GA, mini-swe-agent, and Claude Code as compatibility backends
 4. add more adapters only after the adapter contract has enough real usage

bayesian_agent/harness/llm.py

@@ -1,6 +1,6 @@
 # Architecture
 
-Bayesian-Agent is intentionally small. The framework core is independent from any specific agent harness, so the same Bayesian Skill/SOP evolution loop can support full runs, incremental repair, and cross-harness adaptation.
+Bayesian-Agent is intentionally small. The framework core is independent from any specific external agent harness, so the same Bayesian Skill/SOP evolution loop can support first-party native runs, GenericAgent-backed runs, incremental repair, and cross-harness adaptation.
 
 <div align="center">
   <img src="assets/bayesian_agent_framework_v2.svg" width="900" alt="Bayesian-Agent framework"/>
@@ -11,7 +11,7 @@ Bayesian-Agent is intentionally small. The framework core is independent from an
 ## Data Flow
 
 ```text
-Any Compatible Harness Run
+Native BA Harness or Compatible External Harness
       |
       v
 TrajectoryEvidence
@@ -26,7 +26,7 @@ SkillBelief + RewritePolicy
 SkillContextBuilder
       |
       v
-Adapter
+Native Harness / Adapter
       |
       v
 Any Compatible Harness Next Run
@@ -43,23 +43,35 @@ bayesian_agent/
     policy.py      # default rewrite policy
     context.py     # posterior audit and Skill context rendering
     repair.py      # result normalization and repair summaries
+  harness/
+    native.py      # first-party LLM/tool turn loop
+    llm.py         # OpenAI-compatible chat client
+    tools.py       # workspace-scoped tools
+    core.py        # task envelope, artifacts, three-layer memory bridge
+  memory/
+    layers.py      # hippocampus, intermediate state, cortex
   adapters/
     base.py        # AgentAdapter protocol
+    bayesian_agent.py
     generic_agent.py
+    mini_swe_agent.py
+    claude_code.py
   cli.py
 ```
 
 ## Core Boundaries
 
 `bayesian_agent.core` is framework-agnostic. It knows nothing about GenericAgent, benchmark runners, browser tools, or model APIs.
 
-`bayesian_agent.adapters` defines how external harnesses can connect. The GenericAgent adapter in v0.4 is intentionally a boundary placeholder, not a vendored copy of GenericAgent.
+`bayesian_agent.harness` contains the first-party native harness. It runs the OpenAI-compatible LLM loop, dispatches workspace tools, captures trajectories, and bridges into the three-layer memory system.
+
+`bayesian_agent.adapters` defines how external harnesses can connect. GenericAgent, mini-swe-agent, and Claude Code are optional compatibility backends, not vendored runtimes.
 
 This separation is what prevents Bayesian-Agent from being swallowed by the agent framework category. It is a reusable Bayesian evolution layer that can sit beside multiple harnesses rather than competing with all of them as another monolithic runtime.
 
 `schemas/` defines portable JSON shapes for trajectories and Skill beliefs.
 
-`artifacts/` contains result files from the initial GenericAgent validation. GenericAgent is the current experimental harness; a dedicated Bayesian-Agent harness is planned.
+`artifacts/` contains result files from the initial GenericAgent validation. `results/native_harness_deepseek_v4_flash_full/` and `results/native_harness_deepseek_v4_pro_full/` contain local full-sample results from the first-party native harness.
 
 ## Persistence Model
 

docs/adapters.md

@@ -2,7 +2,7 @@
 
 > 仓库地址：[https://github.com/DataArcTech/Bayesian-Agent](https://github.com/DataArcTech/Bayesian-Agent)  
 > 文档地址：[https://dataarctech.github.io/Bayesian-Agent/](https://dataarctech.github.io/Bayesian-Agent/)  
-> 当前版本：v0.4，arXiv coming soon
+> 当前版本：v0.5，arXiv coming soon
 
 过去一年，Agent 框架越来越多。每个框架都在讲工具调用、memory、planner、browser、workflow、multi-agent，但如果把这些概念拨开，本质问题其实很朴素：
 
@@ -40,7 +40,7 @@ P(success | theta, C, skill)
 
 ## 仓库架构图
 
-下面这张图是当前仓库结构和数据流。核心点是：Bayesian-Agent 的 `core` 不绑定任何具体 Agent runtime；GenericAgent 只是当前实验使用的 backend harness，未来也会上传我们自己的 Agent harness。
+下面这张图是当前仓库结构和数据流。核心点是：Bayesian-Agent 的 `core` 不绑定任何具体 Agent runtime；现在默认可以使用 BA 自家的 native harness，GenericAgent、mini-swe-agent 和 Claude Code 保留为可选兼容 backend。
 
 <div align="center">
   <img src="../assets/bayesian_agent_repository_architecture.svg" width="900" alt="Bayesian-Agent repository architecture"/>
@@ -51,9 +51,11 @@ P(success | theta, C, skill)
 仓库中最重要的几层是：
 
 - `bayesian_agent/core/`：框架无关的 Bayesian evolution engine
+- `bayesian_agent/harness/`：自家的极简 LLM loop、workspace tools 和 trajectory capture
+- `bayesian_agent/memory/`：三层 hippocampus / state / cortex 记忆
 - `bayesian_agent/adapters/`：外部 Agent harness 的适配边界
 - `schemas/`：trajectory evidence 和 Skill belief 的通用 JSON schema
-- `artifacts/`：GenericAgent + deepseek-v4-flash 的实验结果
+- `artifacts/` 和 `results/`：GenericAgent 历史验证与 BA native 全样本/调试实验结果
 - `docs/`：方法、实验、API、adapter 和部署文档
 
 这个边界设计很重要。它意味着 Bayesian-Agent 不需要复制 GenericAgent，也不需要把自己变成一个封闭的大型 Agent 框架。只要某个 Agent framework 能产出统一的 trajectory schema，它就可以接入 Bayesian-Agent。
@@ -105,14 +107,16 @@ Bayesian-Agent 可以从零开始运行任务。没有历史 traces，没有人
 
 **一个 Agent 能不能在没有先验经验的情况下，通过执行和验证自我沉淀 Skill？**
 
-在 SOP-Bench 上，我们用 GenericAgent + `deepseek-v4-flash` 做了原型验证：
+早期原型先用 GenericAgent + `deepseek-v4-flash` 做验证；v0.5 之后，Bayesian-Agent 也已经能用自家的 native harness 跑完整 benchmark：
 
 | Benchmark | Agent | Model | Accuracy | Input Tokens | Output Tokens | Total Tokens | Efficiency |
 |---|---|---|---:|---:|---:|---:|---:|
 | SOP-Bench | GA | deepseek-v4-flash | 80% | 1.34M | 57k | 1.39M | 11.47 |
 | SOP-Bench | GA+Bayesian | deepseek-v4-flash | 100% | 1.07M | 52k | 1.12M | 17.86 |
 
-结果很直观：full mode 不只是把准确率从 80% 拉到 100%，token 使用也从 1.39M 降到了 1.12M。
+结果很直观：GA-backed full mode 不只是把准确率从 80% 拉到 100%，token 使用也从 1.39M 降到了 1.12M。
+
+在 BA native harness 的全样本结果里，`deepseek-v4-flash` 在 SOP-Bench 上从 19/20 baseline 修到 20/20，在 Lifelong AgentBench 上也从 19/20 修到 20/20；RealFin-Bench 上 `deepseek-v4-pro` 从 26/40 baseline 提升到 31/40 final。这里更重要的不是把 harness 写复杂，而是证明 BA 可以自己执行、记录 trajectory，并继续把能力提升交给 Bayesian Skill/SOP evolution。
 
 这说明 Skill 进化不是“越写越长”的记忆堆叠。用 posterior belief 过滤和组织经验，反而可能让上下文更短、更准。
 
@@ -139,8 +143,8 @@ Base Agent -> Failure Traces -> Bayesian Skill Evolution -> Rerun Failures -> Hi
 
 | Benchmark | Agent | Model | Final Accuracy | Incremental Input | Incremental Output | Incremental Total | Incremental Efficiency |
 |---|---|---|---:|---:|---:|---:|---:|
-| SOP-Bench | GA+BayesianIncremental | deepseek-v4-flash | 100% | 216k | 10k | 226k | 17.73 |
-| Lifelong AgentBench | GA+BayesianIncremental | deepseek-v4-flash | 100% | 71k | 7k | 78k | 25.57 |
+| SOP-Bench | GA+BayesianIncremental | deepseek-v4-flash | 100% | 254k | 14k | 268k | 14.93 |
+| Lifelong AgentBench | GA+BayesianIncremental | deepseek-v4-flash | 100% | 129k | 10k | 139k | 14.41 |
 
 这点非常关键：如果一个 Agent 已经能做到 80% 或 90%，Bayesian-Agent 不需要完整重跑所有任务，只需要针对失败部分做增量修复。
 
@@ -150,7 +154,7 @@ Base Agent -> Failure Traces -> Bayesian Skill Evolution -> Rerun Failures -> Hi
 
 Bayesian-Agent 最重要的定位，是 **cross-harness adaptation**。
 
-我们现在的实验基于 GenericAgent，但仓库没有 copy 一份 GenericAgent，也没有把自己写成 GenericAgent fork。原因很简单：
+早期实验基于 GenericAgent，但仓库没有 copy 一份 GenericAgent，也没有把自己写成 GenericAgent fork；现在 BA native、GenericAgent、mini-swe-agent 和 Claude Code 都只是不同执行边界。原因很简单：
 
 **真正有价值的不是某个 harness 本身，而是 Skill/SOP 进化的方法可以跨 harness 复用。**
 
@@ -167,13 +171,13 @@ class AgentAdapter(Protocol):
         ...
 ```
 
-这意味着 GenericAgent、我们后续上传的自研 Agent harness，以及其他 agent frameworks，都可以作为 Bayesian-Agent 的 backend。
+这意味着 BA native、GenericAgent、mini-swe-agent、Claude Code 以及其他 agent frameworks，都可以作为 Bayesian-Agent 的 backend。
 
 我们希望 Bayesian-Agent 成为一个 **Skill evolution substrate**，而不是又一个孤立的 Agent 应用。
 
 ## 仓库里现在有什么？
 
-v0.4 已经包含：
+v0.5 已经包含：
 
 - Bayesian Skill registry
 - Bayesian Evidence Model belief update
@@ -182,9 +186,12 @@ v0.4 已经包含：
 - posterior audit context builder
 - incremental repair utilities
 - result summarization CLI
+- first-party native harness
+- three-layer hippocampus / state / cortex memory
 - GenericAgent optional adapter boundary
+- mini-swe-agent and Claude Code optional backend boundary
 - trajectory 和 skill belief schema
-- SOP-Bench / Lifelong AgentBench 实验 artifacts
+- SOP-Bench / Lifelong AgentBench / RealFin 实验 artifacts
 - MkDocs 文档站和 GitHub Pages 部署
 
 安装方式：
@@ -248,10 +255,10 @@ Bayesian Skill Evolution above
 
 接下来我们会继续做几件事：
 
-- 上传我们自己的 Bayesian-Agent harness，让用户不依赖 GenericAgent 也能完整跑 full loop
+- 继续压缩 native harness 的 token 成本，尤其是 RealFin 这类数据密集任务
 - 完善更多 agent frameworks 的 adapter examples
 - 增加更丰富的 rewrite policy，比如 context-specific split、cost-aware compression、failure taxonomy
-- 加入可执行 benchmark runner，方便外部复现实验
+- 扩展可执行 benchmark runner，方便外部复现实验
 - 推进论文版本，arXiv coming soon
 
 ## 结语