feat: add distributed owner SSD backing tier for KV cache storage

.github/workflows/all_test.yml

@@ -54,6 +54,8 @@ jobs:
             protobuf-compiler \
             python3-venv \
             rsync
+          sudo apt-get clean
+          sudo rm -rf /var/lib/apt/lists/*
 
       - name: Verify Docker availability
         run: |
@@ -87,6 +89,7 @@ jobs:
               --skip-dispatch \
               --skip-start-testbed \
               --skip-runner \
+              --cleanup-pack-runtime-after-success \
               --print-generated
 
       - name: Archive Fluxon CI release
@@ -214,6 +217,7 @@ jobs:
               --suite-path "$RUNNER_TEMP/ci_test_list.ci.yaml" \
               --print-generated \
               --testbed-hostworkdir "$RUNNER_TEMP/fluxon_deploy" \
+              --cleanup-successful-case-artifacts \
               --skip-pack
           rc=$?
           if [ "$rc" -eq 0 ]; then
@@ -225,7 +229,7 @@ jobs:
           exit "$rc"
 
       - name: Normalize ci_2_virt_node debug artifact permissions
-        if: ${{ always() }}
+        if: ${{ failure() }}
         run: |
           chmod -R a+rX .dever/ci_2_virt_node || true
           find .dever/ci_2_virt_node -type f -name '*.yaml' -exec chmod a+r {} + || true
@@ -258,7 +262,7 @@ jobs:
           path: ${{ runner.temp }}/codex-failure-context/**
 
       - name: Upload ci_2_virt_node debug artifacts
-        if: ${{ always() }}
+        if: ${{ failure() }}
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
           name: ci-2-virt-node-debug-${{ github.sha }}

AGENTS.md

@@ -1,5 +1,8 @@
 Keep this document concise.
 - Core user, developer, and design docs are in-repo under fluxon_doc_cn/ and fluxon_doc_en/
+- Detailed bilingual doc writing rules are indexed at `fluxon_doc_en/dev_doc/Developer - 3 - Documentation Writing Rules.md` and `fluxon_doc_cn/dev_doc/开发者 - 3 - 文档写作规约.md`
+- The bilingual technical copy-editing workflow and one-shot example are at `fluxon_doc_en/dev_doc/Developer - 5 - Technical Documentation Copy Editing.md` and `fluxon_doc_cn/dev_doc/开发者 - 5 - 技术文档审校.md`
+- Bilingual code review rules, finding levels, and review templates are at `fluxon_doc_en/dev_doc/Developer - 6 - Code Review Guidelines.md` and `fluxon_doc_cn/dev_doc/开发者 - 6 - Code Review 规约.md`
 - Repository-level rules and component contracts are indexed under Contract Index below. Keep AGENTS summaries concise; put reusable details in linked bilingual docs and update both language indexes together.
 - teststack has two steps: start testbed and testrunner
 - teststack has UI support; testrunner should own the UI authority and API surface, and the UI should run as a long-lived service that reuses the ops interfaces underneath
@@ -9,6 +12,9 @@ Keep this document concise.
 - Git operations are limited to basic `stage`, `unstage`, `commit`, and `push`. Do not use other Git operations.
 - Prefer contraction over compatibility by default. Do not add compatibility layers, deprecated paths, or aliases unless the task explicitly requires them.
 - Prefer one canonical name for one concept. Avoid synonym parameters, duplicated entrypoints, and parallel config surfaces.
+- When a change crosses module boundaries or moves resource cleanup, identify one final-release owner. Keep public layers on contracts, composition layers on ordering, and internal modules on their own state and cleanup; fix the invariant at its owner without adding outer field mutation or a duplicate close path.
+- When a change adds lifecycle dependencies or background work, write the dependency order and implement shutdown as admission stop, scoped wake or cancel, quiescence, dependent release, then dependency release. Keep independent branches parallel and avoid a global lock or coordinator unless the dependency graph requires it.
+- When shutdown intent and cleanup completion can diverge, assign each state a scope and sole writer, keep transitions monotonic, make close repeatable, and treat successful close as a completion barrier. Do not add multiple sources of truth for the same lifecycle fact.
 - Do not add forwarding wrappers that only rename a call, pass through arguments, or unpack/repack its result without adding a contract, validation, transformation, or ownership boundary. Call the canonical implementation directly.
 - Do not use environment variables for ordinary parameter passing. Prefer configuration files first, then explicit command-line arguments.
 - Prefer convention over configuration. When one canonical path or default wiring is sufficient, do not add extra config knobs.
@@ -56,5 +62,6 @@ Keep this document concise.
 ## Contract Index
 
 - Documentation writing: lead with stable conclusions, scope behavioral and performance claims, and keep user/developer docs bilingual by default. See [Developer - 3 - Documentation Writing Rules](<fluxon_doc_en/dev_doc/Developer - 3 - Documentation Writing Rules.md>) and [开发者 - 3 - 文档写作规约](<fluxon_doc_cn/dev_doc/开发者 - 3 - 文档写作规约.md>).
+- Sensitive information: tracked source, configuration, docs, logs, screenshots, fixtures, and generated artifacts must not contain credentials or unapproved real environment identifiers such as IPs, hostnames, storage paths, or cluster topology. Use documentation-reserved placeholders, sanitize output before publication, and treat suspected disclosure as blocking until resolved. See R6 in [Developer - 6 - Code Review Guidelines](<fluxon_doc_en/dev_doc/Developer - 6 - Code Review Guidelines.md>) and [开发者 - 6 - Code Review 规约](<fluxon_doc_cn/dev_doc/开发者 - 6 - Code Review 规约.md>).
 - Tokio async state notification: persistent state is authoritative and `Notify` is a wake-up hint. Ordinary synchronous-predicate waits must use `fluxon_util::notify_state`; custom loops are reserved for added contracts such as blocker diagnostics or timers. A single-future `select!` plus `else` is not a non-blocking poll. See [Developer - 5 - Tokio Notify Usage Rules](<fluxon_doc_en/dev_doc/Developer - 5 - Tokio Notify Usage Rules.md>) and [开发者 - 5 - Tokio Notify 使用规约](<fluxon_doc_cn/dev_doc/开发者 - 5 - Tokio Notify 使用规约.md>).
 - MQ shutdown: user and test paths close every public producer / consumer and consume its `Result` before closing the backing `KvClient`. Endpoint `close()` must finish local runtime, task, keepalive, and handle teardown; lease-backed key deletion is best effort, warns on failure, and falls back to backend TTL. Runtime etcd state transitions keep their strong error contract. Do not access private MQ lifecycle objects, and keep Fluxon KV lease allocation and keepalive in native Rust without Python callbacks. See [User - 4 - MQ Interface](<fluxon_doc_en/user_doc/User - 4 - MQ Interface.md#shutdown-lifecycle>) and [用户 - 4 - MQ 接口](<fluxon_doc_cn/user_doc/用户 - 4 - MQ接口.md#关闭生命周期>).

AGENTS_CN.md

@@ -1,5 +1,8 @@
 保持本文档简洁。
 - 核心用户文档、开发文档和设计文档都在仓库内的 `fluxon_doc_cn/` 和 `fluxon_doc_en/` 下
+- 详细的中英文文档写作规约索引见 `fluxon_doc_cn/dev_doc/开发者 - 3 - 文档写作规约.md` 和 `fluxon_doc_en/dev_doc/Developer - 3 - Documentation Writing Rules.md`
+- 中英文技术文档审校流程和 one-shot 示例见 `fluxon_doc_cn/dev_doc/开发者 - 5 - 技术文档审校.md` 和 `fluxon_doc_en/dev_doc/Developer - 5 - Technical Documentation Copy Editing.md`
+- 中英文 Code Review 规约、finding 等级和 Review 模板见 `fluxon_doc_cn/dev_doc/开发者 - 6 - Code Review 规约.md` 和 `fluxon_doc_en/dev_doc/Developer - 6 - Code Review Guidelines.md`
 - 仓库级规则和组件契约统一收录在下方的“规约索引”中。`AGENTS` 只保留简明摘要；可复用细则写入索引指向的双语文档，并同步更新中英文索引。
 - `teststack` 有两个步骤：`start testbed` 和 `testrunner`
 - `teststack` 支持 UI；`testrunner` 应负责 UI 的 authority 和 API surface，但 UI 应作为常驻服务运行，并复用下层的 ops 接口
@@ -9,6 +12,9 @@
 - Git 操作仅限基础的 `stage`、`unstage`、`commit` 和 `push`。不要使用其他 Git 操作
 - 默认优先收束而不是兼容。除非任务明确要求，否则不要添加兼容层、废弃路径或别名
 - 一个概念优先只保留一个正式名字。避免同义参数、重复入口和并行配置面
+- 当改动跨越模块边界或移动资源清理职责时，必须指定唯一 final-release owner。公共层只依赖契约，组合层只安排顺序，内部模块维护自己的状态和 cleanup；在不变量 owner 处修复，不增加外层字段操作或第二条 close 路径
+- 当改动新增生命周期依赖或后台任务时，必须写明依赖顺序，并按“关闭入口、定向 wake/cancel、建立静默点、释放依赖方、最后释放被依赖方”执行关闭。互不依赖的分支保持并行，依赖图不需要时不引入全局锁或全局 coordinator
+- 当停止意图和清理完成可能分离时，必须为每个状态指定作用域和唯一 writer，保持状态单调、close 可重复，并把 close 成功作为完成屏障。不要为同一个生命周期事实增加多个 source of truth
 - 不要添加只做调用改名、参数透传或结果拆包 / 重包，却不增加契约、校验、转换或所有权边界的转发包装；应直接调用唯一的标准实现
 - 不要用环境变量传递普通参数。优先使用配置文件，其次使用显式命令行参数
 - 优先约定而不是配置。当一个标准路径或默认连接方式已经足够时，不要增加额外配置项
@@ -56,5 +62,6 @@
 ## 规约索引
 
 - 文档写作：先写稳定结论，明确行为和性能结论的作用范围，并默认保持用户文档与开发文档中英文同步。见 [开发者 - 3 - 文档写作规约](<fluxon_doc_cn/dev_doc/开发者 - 3 - 文档写作规约.md>) 和 [Developer - 3 - Documentation Writing Rules](<fluxon_doc_en/dev_doc/Developer - 3 - Documentation Writing Rules.md>)。
+- 敏感信息：纳入版本控制的源码、配置、文档、日志、截图、测试夹具和生成产物不得包含凭证或未经批准公开的真实环境标识，例如 IP、主机名、存储路径和集群拓扑。使用文档保留占位符，发布前完成脱敏；疑似泄露在确认解决前按 blocking 处理。见 [Developer - 6 - Code Review Guidelines](<fluxon_doc_en/dev_doc/Developer - 6 - Code Review Guidelines.md>) 和 [开发者 - 6 - Code Review 规约](<fluxon_doc_cn/dev_doc/开发者 - 6 - Code Review 规约.md>)中的 R6。
 - Tokio 异步状态通知：持久状态是判定依据，`Notify` 只提供唤醒提示。普通同步谓词等待必须调用 `fluxon_util::notify_state`；只有 blocker 诊断、timer 等额外契约才允许保留自定义循环。单 future 的 `select!` 加 `else` 不能用于非阻塞 poll。见 [开发者 - 5 - Tokio Notify 使用规约](<fluxon_doc_cn/dev_doc/开发者 - 5 - Tokio Notify 使用规约.md>) 和 [Developer - 5 - Tokio Notify Usage Rules](<fluxon_doc_en/dev_doc/Developer - 5 - Tokio Notify Usage Rules.md>)。
 - MQ 关闭：用户和测试路径必须先关闭所有公共 producer / consumer 并消费其 `Result`，再关闭底层 `KvClient`。Endpoint `close()` 必须完成本地 runtime、任务、keepalive 和 handle 回收；绑定 lease 的 key 只做尽力删除，失败记 WARN 并由 backend TTL 兜底。运行中的 etcd 状态转换继续使用强错误契约。不要访问 MQ 私有生命周期对象；Fluxon KV lease 的分配与 keepalive 必须留在原生 Rust 内，不得通过 Python 回调桥接。见 [用户 - 4 - MQ 接口](<fluxon_doc_cn/user_doc/用户 - 4 - MQ接口.md#关闭生命周期>) 和 [User - 4 - MQ Interface](<fluxon_doc_en/user_doc/User - 4 - MQ Interface.md#shutdown-lifecycle>)。

README.md

@@ -134,6 +134,12 @@ The `TCP Benchmark` shows that Fluxon outperforms `MooncakeStore` and `Redis` on
 
 ![](./pics/kv_benchmark_chart.png)
 
+Fluxon KV can also use an owner's local SSD as a runtime backing layer for DRAM replicas. The chart below shows a single-node H100 SSD-pressure experiment measured through CUDA event completion. At c16, Fluxon's hit payload throughput for 4/8/16 MiB payloads was 3.83×/5.61×/6.73× that of the faster of the two Mooncake topologies.
+
+![GPU-ready KV throughput and hit rate across Fluxon and Mooncake SSD configurations](./pics/kv_ssd_gpu_sweep.png)
+
+See the Chinese deep dive, [Fluxon KV SSD Storage: Using Local SSD as a Backing Layer for In-Memory Replicas](https://tele-ai.github.io/Fluxon/cn/blog/blog_2_Fluxon_KV_SSD%E5%AD%98%E5%82%A8%EF%BC%9A%E6%8A%8A%E6%9C%AC%E5%9C%B0SSD%E6%8E%A5%E6%88%90%E5%86%85%E5%AD%98%E5%89%AF%E6%9C%AC%E7%9A%84%E5%9B%9E%E5%A1%AB%E5%B1%82), for the full setup, hit rates, and scope.
+
 ### Fluxon FS Benchmark
 
 The benchmark results show that small-file reads and large-file writes already outperform `Alluxio`, large-file read performance is broadly on par, and small-file write performance still has further room to improve.
@@ -294,6 +300,9 @@ Contributions are welcome. Before you start, please read the developer docs on G
 - [Developer - 2 - Package middleware and images](https://tele-ai.github.io/Fluxon/dev_doc/Developer---2---Package-Middleware-and-Images/)
 - [Developer - 3 - Documentation Writing Rules](https://tele-ai.github.io/Fluxon/dev_doc/Developer---3---Documentation-Writing-Rules/)
 - [Developer - 4 - Publish a release](https://tele-ai.github.io/Fluxon/dev_doc/Developer---4---Publish-a-Release/)
+- [Developer - 5 - Technical Documentation Copy Editing](https://tele-ai.github.io/Fluxon/dev_doc/Developer---5---Technical-Documentation-Copy-Editing/)
+- [Developer - 6 - Code Review Guidelines](https://tele-ai.github.io/Fluxon/dev_doc/Developer---6---Code-Review-Guidelines/)
+- [Developer - 7 - Event Subscription and Full Snapshot Guidelines](https://tele-ai.github.io/Fluxon/dev_doc/Developer---7---Event-Subscription-and-Full-Snapshot-Guidelines/)
 
 <a id="contributors"></a>
 

README_CN.md

@@ -136,6 +136,12 @@ Fluxon FS 是一款面向 AI 数据与模型文件、兼容 `S3` 的高性能文
 
 ![](./pics/kv_benchmark_chart.png)
 
+Fluxon KV 还可将 owner 本地 SSD 作为 DRAM 副本的运行期回填层。下图为单机 H100 的 SSD-pressure 实验，以 CUDA event 完成为计数边界；c16 下，Fluxon 在 4/8/16 MiB 组中的 hit payload 吞吐相对两种 Mooncake 拓扑中更快的一种分别达到 3.83×/5.61×/6.73×。
+
+![Fluxon 与 Mooncake 不同 SSD 配置下的 GPU 可用 KV 吞吐和命中率](./pics/kv_ssd_gpu_sweep.png)
+
+完整实验设置、命中率和适用边界见博客：[Fluxon KV SSD 存储：把本地 SSD 接成内存副本的回填层](https://tele-ai.github.io/Fluxon/cn/blog/blog_2_Fluxon_KV_SSD%E5%AD%98%E5%82%A8%EF%BC%9A%E6%8A%8A%E6%9C%AC%E5%9C%B0SSD%E6%8E%A5%E6%88%90%E5%86%85%E5%AD%98%E5%89%AF%E6%9C%AC%E7%9A%84%E5%9B%9E%E5%A1%AB%E5%B1%82)。
+
 ### Fluxon FS 基准测试
 
 测试结果显示，小文件读取和大文件写入性能已显著优于 `Alluxio`，大文件读取性能基本持平，小文件写入性能仍有进一步优化的空间。
@@ -296,6 +302,9 @@ ui
 - [开发者 - 2 - 打包中间件和镜像](https://tele-ai.github.io/Fluxon/cn/dev_doc/%E5%BC%80%E5%8F%91%E8%80%85---2---%E6%89%93%E5%8C%85%E4%B8%AD%E9%97%B4%E4%BB%B6%E5%92%8C%E9%95%9C%E5%83%8F/)
 - [开发者 - 3 - 文档写作规约](https://tele-ai.github.io/Fluxon/cn/dev_doc/%E5%BC%80%E5%8F%91%E8%80%85---3---%E6%96%87%E6%A1%A3%E5%86%99%E4%BD%9C%E8%A7%84%E7%BA%A6/)
 - [开发者 - 4 - 发布 Release](https://tele-ai.github.io/Fluxon/cn/dev_doc/%E5%BC%80%E5%8F%91%E8%80%85---4---%E5%8F%91%E5%B8%83-Release/)
+- [开发者 - 5 - 技术文档审校](https://tele-ai.github.io/Fluxon/cn/dev_doc/%E5%BC%80%E5%8F%91%E8%80%85---5---%E6%8A%80%E6%9C%AF%E6%96%87%E6%A1%A3%E5%AE%A1%E6%A0%A1/)
+- [开发者 - 6 - Code Review 规约](https://tele-ai.github.io/Fluxon/cn/dev_doc/%E5%BC%80%E5%8F%91%E8%80%85---6---Code-Review-%E8%A7%84%E7%BA%A6/)
+- [开发者 - 7 - 事件订阅与全量快照规约](https://tele-ai.github.io/Fluxon/cn/dev_doc/%E5%BC%80%E5%8F%91%E8%80%85---7---%E4%BA%8B%E4%BB%B6%E8%AE%A2%E9%98%85%E4%B8%8E%E5%85%A8%E9%87%8F%E5%BF%AB%E7%85%A7%E8%A7%84%E7%BA%A6/)
 
 <a id="contributors"></a>
 

deployment/gen_k8s_daemonset.py

@@ -349,7 +349,7 @@ def _mirror_daemonset_yaml_outdir(*, src_outdir: str, dst_outdir: str) -> None:
     #
     # Causal chain:
     # - Operators often run the generator from the repo workdir, while fluxon-deployer reads
-    #   manifests from a hostworkdir-mounted directory (e.g. /opt/store_team_dev/fluxon_deployer).
+    #   manifests from a hostworkdir-mounted directory (e.g. /tmp/fluxon-example/deploy).
     # - If the two directories drift, deployer applies stale manifests and can crash-loop.
     # - Therefore we provide an explicit opt-in "mirror_outdir" that:
     #   - cleans the destination directory (same semantics as the primary outdir)

deployment/tests/test_build_doc_site_contract.py

@@ -99,6 +99,16 @@ def test_rewrite_homepage_target_path_for_chinese_home(self) -> None:
             "../fluxon_rs/rust-toolchain.toml",
         )
 
+    def test_rewrite_variant_repo_image_for_quartz_route(self) -> None:
+        for language in ("en", "cn"):
+            with self.subTest(language=language):
+                self.assertEqual(
+                    _DOC_SITE.rewrite_variant_target_path(
+                        "../../pics/kv_ssd_cpu_sweep.png", language=language
+                    ),
+                    "../pics/kv_ssd_cpu_sweep.png",
+                )
+
     def test_stage_cn_only_design_into_en_root_when_english_design_missing(self) -> None:
         tmpdir = Path(tempfile.mkdtemp(prefix="fluxon_doc_site_contract_"))
         old_stage_root = _DOC_SITE.STAGE_DOCS_ROOT