Skip to content

Commit ea58bc2

Browse files
loningclaude
andauthored
test 模式统一外部命令 mock(fkst.test.mock_command/command_calls) (#9)
引擎对外部 CLI 的调用(exec_sync 的 /bin/sh、spawn_codex_sync 的 codex、 git_* 原语的 git)此前在测试里只能靠每包自写 fake 二进制 on PATH,per-tool、 per-package、模拟 CLA。本改在 test 模式提供统一劫持:按渲染命令行匹配、返回 预设结果、未 mock fail-closed。生产路径完全不变。 - 新增 crates/fkst-framework/src/external_command.rs:MockCommandState(Arc, FIFO mock 队列 + call 记录)+ 渲染命令行(exec_sync=shell cmd 串;codex= codex exec ... -;git=git -C <root> ...)。 - 三执行点(sdk_basic exec_sync、sdk_git、sdk_codex)在真进程前检查注入的 mock runner:test 命中返回 {stdout,stderr,exit_code}(exec 直接返回 / git 照常解析 mocked stdout / codex 短路返回);未命中 → Lua error "unmocked external command: <rendered>",绝不 Command::new。codex 仅加 test-mode 短路分支,未重构 streaming/permit/stall/log 真路径。 setup_worktree 的 git 调用经同一 runner(test 未 mock fail-closed),不合成 worktree 副作用。 - fkst.test.mock_command(pattern, {stdout,stderr?,exit_code?}):前缀/子串匹配 渲染命令行,FIFO 一次性消费,多注册按序。fkst.test.command_calls() 返回 已记录调用(含 codex prompt/stdin)供断言。 - 注入:register_test_sdk 建共享 Arc;run_department dept_lua 共享同一 Arc; 每 test function 前 reset。生产 register_framework_sdk 传 None、行为不变; supervise/run/self-test/conformance 无 mock。 - primitive 自测(sdk_basic/sdk_codex/sdk_git)保留真工具/fake-PATH,不迁 mock (仅 +2 行 #[path] mod external_command)。 - 文档:SPEC/CLAUDE/README/architecture/examples 同步 test-mode mock 说明。 测试:test_runner_cli 含 14 个 mock 场景(exec/codex sync+async/git 读原语经 mocked stdout 解析/fail-closed 各类/per-test 隔离/FIFO 序列消费/command_calls stdin/setup_worktree fail-closed);bin 116、self_test 6、sdk_git serial 26、 supervise_smoke 5 全绿。 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent dcec900 commit ea58bc2

16 files changed

Lines changed: 662 additions & 82 deletions

File tree

CLAUDE.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,8 @@ Department execution 由 supervise spawn `fkst-framework run <lua> --project-roo
106106

107107
Codex 调用固定为 `codex exec --dangerously-bypass-approvals-and-sandbox [-C worktree] [--context context] -`。prompt 写入 stdin;stdin EOF 是调用边界。stdout、stderr、exit_code、cmd、done time、stall window 必须写入 codex log。`spawn_codex` 返回的 handle 只能由同一 pipeline 的 `await_all` 消费,不能跨 pipeline 复用。
108108

109+
`fkst-framework test` 注册 test-mode-only `fkst.test.mock_command(pattern, result)``fkst.test.command_calls()`,和 `run_department` 并列。test mode 劫持 `exec_sync`、codex SDK 与 git SDK 的外部命令调用,按渲染命令行前缀或子串匹配 mock,按注册顺序一次性消费;未 mock 的外部命令 fail closed,不启动真实进程。production `run``supervise``--self-test` 与 conformance 不注册 mock state。`setup_worktree` 在 test mode 通过同一 git mock runner fail closed,但不模拟 worktree 副作用。
110+
109111
## 单 repo 单实例
110112

111113
一个 host git repo 对应一个 supervisor、一个 framework binary、一组 package+host composed graph 和一个 `FKST_RUNTIME_ROOT`。多 repo 或多业务是多次部署,不是在同一 framework 进程里跑多套主链路。

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,9 @@ consumer 的完整事件日志会落在 `<RT>/logs/framework-child/` 下。真
111111

112112
Lua 单元测试由 `fkst-framework test` 执行。runner 只发现 package root 和 host root 下的 `departments/*/*_test.lua``tests/*_test.lua`,不全树递归,也不扫描 `raisers/``fkst/`。测试文件应 `return { test_name = function() ... end }`;runner 按文件路径和 `test_*` key 排序,失败后继续执行后续测试,最后输出通过 / 失败汇总。
113113

114-
`fkst.test` 只在 `test` 子命令的 Lua state 中注册,不属于 production Lua SDK surface;`run``supervise` 模式不可依赖它。当前断言只有 `eq(actual, expected[, msg])``is_true(value[, msg])``raises(fn[, msg])``is_nil(value[, msg])`。test-mode 还提供 `run_department(path, event[, opts])`,用 fresh Lua state、production SDK 和独立 raise buffer 执行一个 department entrypoint,返回 `{ exit_code = int, raises = { { queue = string, payload = table }, ... } }`;queue 解析与 production 一致。每个测试文件按所属 graph root 隔离执行,相对路径按该测试文件所属 owner package root 解析,`opts.cwd``opts.env``opts.path_prepend` 只作用于该次执行并随后恢复。这是最小单测工具,不提供 fixture、mock、hook 或测试框架 DSL;除非有意验证真实 CLI 路径,Lua 单测不应调用 `spawn_codex_sync`
114+
`fkst.test` 只在 `test` 子命令的 Lua state 中注册,不属于 production Lua SDK surface;`run``supervise` 模式不可依赖它。当前断言只有 `eq(actual, expected[, msg])``is_true(value[, msg])``raises(fn[, msg])``is_nil(value[, msg])`。test-mode 还提供 `run_department(path, event[, opts])`,用 fresh Lua state、production SDK 和独立 raise buffer 执行一个 department entrypoint,返回 `{ exit_code = int, raises = { { queue = string, payload = table }, ... } }`;queue 解析与 production 一致。每个测试文件按所属 graph root 隔离执行,相对路径按该测试文件所属 owner package root 解析,`opts.cwd``opts.env``opts.path_prepend` 只作用于该次执行并随后恢复。
115+
116+
`fkst.test.mock_command(pattern, result)` 劫持 test mode 中的 `exec_sync`、codex SDK 与 git SDK 外部命令调用;渲染命令行按前缀或子串匹配,mock 按注册顺序一次性消费。`result``{ stdout = "", stderr = "", exit_code = 0 }` 形状,`stderr``exit_code` 可省略。未 mock 的外部命令 fail closed 且不启动真实进程。`fkst.test.command_calls()` 返回已记录调用,包含渲染命令、program、args、stdin、stdout、stderr 与 exit_code。`setup_worktree` 在 test mode 也通过 git mock runner,但 mock 不合成 worktree 副作用。
115117

116118
production Lua SDK 包含 `once(key, fn) -> boolean`。它是 best-effort per-key de-bounce scratch marker,不是 durable state。`key` 必须是非空相对 filesystem path,`/` 表示目录;每个 segment 非空、匹配 `[A-Za-z0-9._-]+`,且不是 `.``..`;禁止 leading / trailing `/``//`、反斜杠、NUL 与绝对路径。framework 直接使用校验后的 key,在 `<RT>/locks/once/<key>` 上获取 exclusive flock,再检查 `<RT>/marks/<key>``locks/once/` 是 once 内部锁的保留子目录,不属于 `with_lock` 用户锁命名空间。marker 已存在时返回 `false` 且不调用 `fn`;marker 不存在时调用 `fn`,成功后写入 marker 并返回 `true``fn` 失败时错误原样传播且不写 marker,后续调用会重试。
117119

SPEC.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@
6565
- `cache_set(key, value)``cache_get(key)` 读写 `<RT>/cache/<key>``cache_set` 原子覆盖写入 string value;`cache_get` 命中时返回 string,缺失时返回 nil。cache 是 host-local best-effort scratch,不是 durable state;调用者需要 read-compare-write 原子性时必须外层使用 `with_lock`
6666
- Department 默认以可靠方式消费队列;`M.spec.ephemeral = {"queue"}` 可将本 Department 对指定 consumed queue 的订阅降级为非可靠。`M.spec.retry = false` 只表示失败不重试;`M.spec.retry = { ... }` 可覆盖 `max_attempts`、`base`、`cap` 的任意子集,缺失字段从全局默认补齐。可靠订阅启动必须有 `FKST_DURABLE_ROOT`,缺失 fail-closed。可靠 source event 必须带 `SourceRef{kind,reference}`;cron 由 raiser 名派生,file_watch 由绝对路径派生,Department `RAISED` 进入可靠 queue 时继承上游 source_ref,缺失则 publish fail-closed 且上游 delivery 不 ack。可靠 consumer 由 Fanout wake + 定时 tick 调用 redb store `lease`,spawn framework 后仅在 exit 0 且 RAISED publish 成功时 `ack`;失败、stall、spawn error 或 RAISED publish 失败调用 `retry`,到 max attempts 写 redb dead 表并 best-effort publish `dead_letter`。当前 delivery 来自 `dead_letter` 时抑制再次发送 `dead_letter`。该机制不是新 source kind,不提供 exactly-once;语义是 at-least-once-until-ack,`Fanout::send` 在可靠路径只作进程内唤醒。
6767
- `spawn_codex` handle 只能由 `await_all` join;单 handle 等待使用 `await_all({handle})`;first-result fanout 与 sleep timer 不是固定 Lua SDK surface。
68+
- `fkst-framework test` 额外注册 test-mode-only `fkst.test` table。除断言与 `run_department` 外,`fkst.test.mock_command(pattern, result)``fkst.test.command_calls()` 可劫持 `exec_sync`、codex SDK 与 git SDK 的外部命令调用;匹配基于渲染命令行的前缀或子串,mock 按注册顺序一次性消费,未 mock 的外部命令 fail closed 且不启动真实进程。production `run``supervise``--self-test` 与 conformance 不注册该 mock state。`setup_worktree` 在 test mode 也经同一 git mock runner,但不模拟 worktree filesystem 副作用。
6869
- `json` 是 decode-only:`json.decode` 暴露 engine 自身 JSON wire format 的解析(event 进、`raise` 出都是 JSON),Lua 值经 `raise` 出引擎,故不提供 `json.encode``json` table 锁定为只含 `decode`,新增 encode 或其它 key 必须另走 evidence + conformance。
6970
- 新增 SDK 函数必须经 evidence、深度共识与 conformance 覆盖,不能由单个 codex 实例直接扩张。
7071

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
use std::sync::{Arc, Mutex};
2+
3+
#[derive(Clone, Debug)]
4+
pub(crate) struct MockCommandState {
5+
inner: Arc<Mutex<MockCommandInner>>,
6+
}
7+
8+
#[derive(Debug, Default)]
9+
struct MockCommandInner {
10+
mocks: Vec<MockCommand>,
11+
calls: Vec<MockCommandCall>,
12+
}
13+
14+
#[derive(Clone, Debug)]
15+
struct MockCommand {
16+
pattern: String,
17+
result: MockCommandResult,
18+
}
19+
20+
#[derive(Clone, Debug)]
21+
pub(crate) struct MockCommandResult {
22+
pub(crate) stdout: String,
23+
pub(crate) stderr: String,
24+
pub(crate) exit_code: i32,
25+
}
26+
27+
#[derive(Clone, Debug)]
28+
pub(crate) struct MockCommandCall {
29+
pub(crate) rendered: String,
30+
pub(crate) program: String,
31+
pub(crate) args: Vec<String>,
32+
pub(crate) stdin: String,
33+
pub(crate) stdout: String,
34+
pub(crate) stderr: String,
35+
pub(crate) exit_code: i32,
36+
}
37+
38+
impl MockCommandState {
39+
pub(crate) fn new() -> Self {
40+
Self {
41+
inner: Arc::new(Mutex::new(MockCommandInner::default())),
42+
}
43+
}
44+
45+
pub(crate) fn reset(&self) -> mlua::Result<()> {
46+
let mut inner = self.lock()?;
47+
inner.mocks.clear();
48+
inner.calls.clear();
49+
Ok(())
50+
}
51+
52+
pub(crate) fn push_mock(&self, pattern: String, result: MockCommandResult) -> mlua::Result<()> {
53+
let mut inner = self.lock()?;
54+
inner.mocks.push(MockCommand { pattern, result });
55+
Ok(())
56+
}
57+
58+
pub(crate) fn calls(&self) -> mlua::Result<Vec<MockCommandCall>> {
59+
let inner = self.lock()?;
60+
Ok(inner.calls.clone())
61+
}
62+
63+
pub(crate) fn execute(
64+
&self,
65+
rendered: String,
66+
program: String,
67+
args: Vec<String>,
68+
stdin: String,
69+
) -> mlua::Result<MockCommandResult> {
70+
let mut inner = self.lock()?;
71+
let index = inner
72+
.mocks
73+
.iter()
74+
.position(|mock| {
75+
rendered.starts_with(&mock.pattern) || rendered.contains(&mock.pattern)
76+
})
77+
.ok_or_else(|| {
78+
mlua::Error::external(format!("unmocked external command: {rendered}"))
79+
})?;
80+
let mock = inner.mocks.remove(index);
81+
inner.calls.push(MockCommandCall {
82+
rendered,
83+
program,
84+
args,
85+
stdin,
86+
stdout: mock.result.stdout.clone(),
87+
stderr: mock.result.stderr.clone(),
88+
exit_code: mock.result.exit_code,
89+
});
90+
Ok(mock.result)
91+
}
92+
93+
fn lock(&self) -> mlua::Result<std::sync::MutexGuard<'_, MockCommandInner>> {
94+
self.inner
95+
.lock()
96+
.map_err(|_| mlua::Error::external("mock command state lock is poisoned"))
97+
}
98+
}
99+
100+
pub(crate) fn format_command(program: &str, args: &[String]) -> String {
101+
std::iter::once(program.to_string())
102+
.chain(args.iter().map(|arg| shell_quote(arg)))
103+
.collect::<Vec<_>>()
104+
.join(" ")
105+
}
106+
107+
fn shell_quote(value: &str) -> String {
108+
if value
109+
.bytes()
110+
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'_' | b'-' | b'.' | b'/' | b':' | b'='))
111+
{
112+
value.to_string()
113+
} else {
114+
format!("'{}'", value.replace('\'', "'\"'\"'"))
115+
}
116+
}

crates/fkst-framework/src/main.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ use serde_json::Value as JsonValue;
1818
use std::path::PathBuf;
1919

2020
mod config_registry;
21+
mod external_command;
2122
mod host_conformance;
2223
mod mlua_init;
2324
mod path_resolver;

crates/fkst-framework/src/mlua_init.rs

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ use serde_json::Value as JsonValue;
66
use std::path::Path;
77

88
use crate::config_registry::ConfigContext;
9+
use crate::external_command::MockCommandState;
910
use crate::path_resolver::{package_root_path, NameResolver};
1011
use crate::raise::RaiseBuffer;
1112

@@ -35,6 +36,27 @@ pub fn register_framework_sdk(
3536
Ok(())
3637
}
3738

39+
pub(crate) fn register_framework_sdk_with_runner(
40+
lua: &Lua,
41+
raise_buf: RaiseBuffer,
42+
host_root: &Path,
43+
resolver: NameResolver,
44+
owner_namespace: String,
45+
runner: Option<MockCommandState>,
46+
) -> mlua::Result<()> {
47+
let config = ConfigContext::from_host_root(host_root).map_err(mlua::Error::external)?;
48+
crate::sdk_log::register(lua)?;
49+
crate::sdk_basic::register_with_runner(lua, runner.clone())?;
50+
crate::sdk_fs::register(lua)?;
51+
crate::sdk_json::register(lua)?;
52+
crate::sdk_git::register_with_runner(lua, host_root, config.clone(), runner.clone())?;
53+
crate::sdk_mark::register(lua, host_root)?;
54+
crate::sdk_cache::register(lua, host_root)?;
55+
crate::sdk_codex::register_with_runner(lua, host_root, config, runner)?;
56+
crate::raise::register(lua, raise_buf, resolver, owner_namespace)?;
57+
Ok(())
58+
}
59+
3860
/// Convert serde_json::Value to mlua::Value via LuaSerdeExt.
3961
pub fn json_to_lua(lua: &Lua, v: &JsonValue) -> mlua::Result<LuaValue> {
4062
lua.to_value(v)

crates/fkst-framework/src/sdk_basic.rs

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,8 @@ use std::process::{Command, Stdio};
99
use std::thread::JoinHandle;
1010
use std::time::{Duration, Instant};
1111

12+
use crate::external_command::MockCommandState;
13+
1214
struct ExecOptions {
1315
cmd: String,
1416
cwd: Option<String>,
@@ -25,6 +27,10 @@ struct ExecResult {
2527

2628
// Lua SDK registration and self-test match the fixed CLAUDE.md surface exactly; human notification, if needed, is represented through existing git/fs/log facts rather than a new SDK function.
2729
pub fn register(lua: &Lua) -> Result<()> {
30+
register_with_runner(lua, None)
31+
}
32+
33+
pub(crate) fn register_with_runner(lua: &Lua, runner: Option<MockCommandState>) -> Result<()> {
2834
lua.globals().set(
2935
"now",
3036
lua.create_function(|_, ()| {
@@ -38,9 +44,9 @@ pub fn register(lua: &Lua) -> Result<()> {
3844

3945
lua.globals().set(
4046
"exec_sync",
41-
lua.create_function(|lua, arg: Value| {
47+
lua.create_function(move |lua, arg: Value| {
4248
let opts = parse_exec_options(arg)?;
43-
let out = run_exec_sync(opts)?;
49+
let out = run_exec_sync(opts, runner.as_ref())?;
4450
let t = lua.create_table()?;
4551
t.set("stdout", out.stdout)?;
4652
t.set("stderr", out.stderr)?;
@@ -123,7 +129,22 @@ fn kill_process_group(child_pid: u32) {
123129
#[cfg(not(unix))]
124130
fn kill_process_group(_child_pid: u32) {}
125131

126-
fn run_exec_sync(opts: ExecOptions) -> Result<ExecResult> {
132+
fn run_exec_sync(opts: ExecOptions, runner: Option<&MockCommandState>) -> Result<ExecResult> {
133+
if let Some(runner) = runner {
134+
let result = runner.execute(
135+
opts.cmd.clone(),
136+
"/bin/sh".to_string(),
137+
vec!["-c".to_string(), opts.cmd],
138+
String::new(),
139+
)?;
140+
return Ok(ExecResult {
141+
stdout: result.stdout,
142+
stderr: result.stderr,
143+
exit_code: result.exit_code,
144+
timed_out: None,
145+
});
146+
}
147+
127148
match opts.timeout {
128149
Some(timeout) => run_exec_sync_with_timeout(&opts, timeout),
129150
None => {

0 commit comments

Comments
 (0)