From 3df7865f17765a1ac27c9d65fc7565b2d987553b Mon Sep 17 00:00:00 2001 From: auric Date: Fri, 5 Jun 2026 15:47:48 +0800 Subject: [PATCH] =?UTF-8?q?test=20=E6=A8=A1=E5=BC=8F=E7=BB=9F=E4=B8=80?= =?UTF-8?q?=E5=A4=96=E9=83=A8=E5=91=BD=E4=BB=A4=20mock(fkst.test.mock=5Fco?= =?UTF-8?q?mmand/command=5Fcalls)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 引擎对外部 CLI 的调用(exec_sync 的 /bin/sh、spawn_codex_sync 的 codex、 git_* 原语的 git)此前在测试里只能靠每包自写 fake 二进制 on PATH,per-tool、 per-package、模拟 CLA。本改在 test 模式提供统一劫持:按渲染命令行匹配、返回 预设结果、未 mock fail-closed。生产路径完全不变。 - 新增 crates/fkst-framework/src/external_command.rs:MockCommandState(Arc, FIFO mock 队列 + call 记录)+ 渲染命令行(exec_sync=shell cmd 串;codex= codex exec ... -;git=git -C ...)。 - 三执行点(sdk_basic exec_sync、sdk_git、sdk_codex)在真进程前检查注入的 mock runner:test 命中返回 {stdout,stderr,exit_code}(exec 直接返回 / git 照常解析 mocked stdout / codex 短路返回);未命中 → Lua error "unmocked external command: ",绝不 Command::new。codex 仅加 test-mode 短路分支,未重构 streaming/permit/stall/log 真路径。 setup_worktree 的 git 调用经同一 runner(test 未 mock fail-closed),不合成 worktree 副作用。 - fkst.test.mock_command(pattern, {stdout,stderr?,exit_code?}):前缀/子串匹配 渲染命令行,FIFO 一次性消费,多注册按序。fkst.test.command_calls() 返回 已记录调用(含 codex prompt/stdin)供断言。 - 注入:register_test_sdk 建共享 Arc;run_department dept_lua 共享同一 Arc; 每 test function 前 reset。生产 register_framework_sdk 传 None、行为不变; supervise/run/self-test/conformance 无 mock。 - primitive 自测(sdk_basic/sdk_codex/sdk_git)保留真工具/fake-PATH,不迁 mock (仅 +2 行 #[path] mod external_command)。 - 文档:SPEC/CLAUDE/README/architecture/examples 同步 test-mode mock 说明。 测试:test_runner_cli 含 14 个 mock 场景(exec/codex sync+async/git 读原语经 mocked stdout 解析/fail-closed 各类/per-test 隔离/FIFO 序列消费/command_calls stdin/setup_worktree fail-closed);bin 116、self_test 6、sdk_git serial 26、 supervise_smoke 5 全绿。 Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 2 + README.md | 4 +- SPEC.md | 1 + crates/fkst-framework/src/external_command.rs | 116 ++++++++++ crates/fkst-framework/src/main.rs | 1 + crates/fkst-framework/src/mlua_init.rs | 22 ++ crates/fkst-framework/src/sdk_basic.rs | 27 ++- crates/fkst-framework/src/sdk_codex.rs | 105 ++++++--- crates/fkst-framework/src/sdk_git.rs | 191 +++++++++++++---- crates/fkst-framework/src/test_runner.rs | 61 +++++- crates/fkst-framework/tests/sdk_codex.rs | 2 + crates/fkst-framework/tests/sdk_git.rs | 2 + .../fkst-framework/tests/test_runner_cli.rs | 202 ++++++++++++++++++ docs/architecture.md | 4 +- examples/codex-package/README.md | 2 +- examples/minimal-package/README.md | 2 +- 16 files changed, 662 insertions(+), 82 deletions(-) create mode 100644 crates/fkst-framework/src/external_command.rs diff --git a/CLAUDE.md b/CLAUDE.md index bb04eb8..23e7aff 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -106,6 +106,8 @@ Department execution 由 supervise spawn `fkst-framework run --project-roo Codex 调用固定为 `codex exec --dangerously-bypass-approvals-and-sandbox [-C worktree] [--context context] -`。prompt 写入 stdin；stdin EOF 是调用边界。stdout、stderr、exit_code、cmd、done time、stall window 必须写入 codex log。`spawn_codex` 返回的 handle 只能由同一 pipeline 的 `await_all` 消费，不能跨 pipeline 复用。 +`fkst-framework test` 注册 test-mode-only `fkst.test.mock_command(pattern, result)` 与 `fkst.test.command_calls()`，和 `run_department` 并列。test mode 劫持 `exec_sync`、codex SDK 与 git SDK 的外部命令调用，按渲染命令行前缀或子串匹配 mock，按注册顺序一次性消费；未 mock 的外部命令 fail closed，不启动真实进程。production `run`、`supervise`、`--self-test` 与 conformance 不注册 mock state。`setup_worktree` 在 test mode 通过同一 git mock runner fail closed，但不模拟 worktree 副作用。 + ## 单 repo 单实例 一个 host git repo 对应一个 supervisor、一个 framework binary、一组 package+host composed graph 和一个 `FKST_RUNTIME_ROOT`。多 repo 或多业务是多次部署，不是在同一 framework 进程里跑多套主链路。 diff --git a/README.md b/README.md index 4e5d502..fed8816 100644 --- a/README.md +++ b/README.md @@ -111,7 +111,9 @@ consumer 的完整事件日志会落在 `/logs/framework-child/` 下。真 Lua 单元测试由 `fkst-framework test` 执行。runner 只发现 package root 和 host root 下的 `departments/*/*_test.lua` 与 `tests/*_test.lua`，不全树递归，也不扫描 `raisers/` 或 `fkst/`。测试文件应 `return { test_name = function() ... end }`；runner 按文件路径和 `test_*` key 排序，失败后继续执行后续测试，最后输出通过 / 失败汇总。 -`fkst.test` 只在 `test` 子命令的 Lua state 中注册，不属于 production Lua SDK surface；`run` 与 `supervise` 模式不可依赖它。当前断言只有 `eq(actual, expected[, msg])`、`is_true(value[, msg])`、`raises(fn[, msg])` 和 `is_nil(value[, msg])`。test-mode 还提供 `run_department(path, event[, opts])`，用 fresh Lua state、production SDK 和独立 raise buffer 执行一个 department entrypoint，返回 `{ exit_code = int, raises = { { queue = string, payload = table }, ... } }`；queue 解析与 production 一致。每个测试文件按所属 graph root 隔离执行，相对路径按该测试文件所属 owner package root 解析，`opts.cwd`、`opts.env`、`opts.path_prepend` 只作用于该次执行并随后恢复。这是最小单测工具，不提供 fixture、mock、hook 或测试框架 DSL；除非有意验证真实 CLI 路径，Lua 单测不应调用 `spawn_codex_sync`。 +`fkst.test` 只在 `test` 子命令的 Lua state 中注册，不属于 production Lua SDK surface；`run` 与 `supervise` 模式不可依赖它。当前断言只有 `eq(actual, expected[, msg])`、`is_true(value[, msg])`、`raises(fn[, msg])` 和 `is_nil(value[, msg])`。test-mode 还提供 `run_department(path, event[, opts])`，用 fresh Lua state、production SDK 和独立 raise buffer 执行一个 department entrypoint，返回 `{ exit_code = int, raises = { { queue = string, payload = table }, ... } }`；queue 解析与 production 一致。每个测试文件按所属 graph root 隔离执行，相对路径按该测试文件所属 owner package root 解析，`opts.cwd`、`opts.env`、`opts.path_prepend` 只作用于该次执行并随后恢复。 + +`fkst.test.mock_command(pattern, result)` 劫持 test mode 中的 `exec_sync`、codex SDK 与 git SDK 外部命令调用；渲染命令行按前缀或子串匹配，mock 按注册顺序一次性消费。`result` 是 `{ stdout = "", stderr = "", exit_code = 0 }` 形状，`stderr` 与 `exit_code` 可省略。未 mock 的外部命令 fail closed 且不启动真实进程。`fkst.test.command_calls()` 返回已记录调用，包含渲染命令、program、args、stdin、stdout、stderr 与 exit_code。`setup_worktree` 在 test mode 也通过 git mock runner，但 mock 不合成 worktree 副作用。 production Lua SDK 包含 `once(key, fn) -> boolean`。它是 best-effort per-key de-bounce scratch marker，不是 durable state。`key` 必须是非空相对 filesystem path，`/` 表示目录；每个 segment 非空、匹配 `[A-Za-z0-9._-]+`，且不是 `.` 或 `..`；禁止 leading / trailing `/`、`//`、反斜杠、NUL 与绝对路径。framework 直接使用校验后的 key，在 `/locks/once/` 上获取 exclusive flock，再检查 `/marks/`。`locks/once/` 是 once 内部锁的保留子目录，不属于 `with_lock` 用户锁命名空间。marker 已存在时返回 `false` 且不调用 `fn`；marker 不存在时调用 `fn`，成功后写入 marker 并返回 `true`；`fn` 失败时错误原样传播且不写 marker，后续调用会重试。 diff --git a/SPEC.md b/SPEC.md index 23a9b01..a935f05 100644 --- a/SPEC.md +++ b/SPEC.md @@ -65,6 +65,7 @@ - `cache_set(key, value)` 与 `cache_get(key)` 读写 `/cache/`。`cache_set` 原子覆盖写入 string value；`cache_get` 命中时返回 string，缺失时返回 nil。cache 是 host-local best-effort scratch，不是 durable state；调用者需要 read-compare-write 原子性时必须外层使用 `with_lock`。 - Department 默认以可靠方式消费队列；`M.spec.ephemeral = {"queue"}` 可将本 Department 对指定 consumed queue 的订阅降级为非可靠。`M.spec.retry = false` 只表示失败不重试；`M.spec.retry = { ... }` 可覆盖 `max_attempts`、`base`、`cap` 的任意子集，缺失字段从全局默认补齐。可靠订阅启动必须有 `FKST_DURABLE_ROOT`，缺失 fail-closed。可靠 source event 必须带 `SourceRef{kind,reference}`；cron 由 raiser 名派生，file_watch 由绝对路径派生，Department `RAISED` 进入可靠 queue 时继承上游 source_ref，缺失则 publish fail-closed 且上游 delivery 不 ack。可靠 consumer 由 Fanout wake + 定时 tick 调用 redb store `lease`，spawn framework 后仅在 exit 0 且 RAISED publish 成功时 `ack`；失败、stall、spawn error 或 RAISED publish 失败调用 `retry`，到 max attempts 写 redb dead 表并 best-effort publish `dead_letter`。当前 delivery 来自 `dead_letter` 时抑制再次发送 `dead_letter`。该机制不是新 source kind，不提供 exactly-once；语义是 at-least-once-until-ack，`Fanout::send` 在可靠路径只作进程内唤醒。 - `spawn_codex` handle 只能由 `await_all` join；单 handle 等待使用 `await_all({handle})`；first-result fanout 与 sleep timer 不是固定 Lua SDK surface。 +- `fkst-framework test` 额外注册 test-mode-only `fkst.test` table。除断言与 `run_department` 外，`fkst.test.mock_command(pattern, result)` 与 `fkst.test.command_calls()` 可劫持 `exec_sync`、codex SDK 与 git SDK 的外部命令调用；匹配基于渲染命令行的前缀或子串，mock 按注册顺序一次性消费，未 mock 的外部命令 fail closed 且不启动真实进程。production `run`、`supervise`、`--self-test` 与 conformance 不注册该 mock state。`setup_worktree` 在 test mode 也经同一 git mock runner，但不模拟 worktree filesystem 副作用。 - `json` 是 decode-only：`json.decode` 暴露 engine 自身 JSON wire format 的解析（event 进、`raise` 出都是 JSON），Lua 值经 `raise` 出引擎，故不提供 `json.encode`；`json` table 锁定为只含 `decode`，新增 encode 或其它 key 必须另走 evidence + conformance。 - 新增 SDK 函数必须经 evidence、深度共识与 conformance 覆盖，不能由单个 codex 实例直接扩张。 diff --git a/crates/fkst-framework/src/external_command.rs b/crates/fkst-framework/src/external_command.rs new file mode 100644 index 0000000..32841e3 --- /dev/null +++ b/crates/fkst-framework/src/external_command.rs @@ -0,0 +1,116 @@ +use std::sync::{Arc, Mutex}; + +#[derive(Clone, Debug)] +pub(crate) struct MockCommandState { + inner: Arc>, +} + +#[derive(Debug, Default)] +struct MockCommandInner { + mocks: Vec, + calls: Vec, +} + +#[derive(Clone, Debug)] +struct MockCommand { + pattern: String, + result: MockCommandResult, +} + +#[derive(Clone, Debug)] +pub(crate) struct MockCommandResult { + pub(crate) stdout: String, + pub(crate) stderr: String, + pub(crate) exit_code: i32, +} + +#[derive(Clone, Debug)] +pub(crate) struct MockCommandCall { + pub(crate) rendered: String, + pub(crate) program: String, + pub(crate) args: Vec, + pub(crate) stdin: String, + pub(crate) stdout: String, + pub(crate) stderr: String, + pub(crate) exit_code: i32, +} + +impl MockCommandState { + pub(crate) fn new() -> Self { + Self { + inner: Arc::new(Mutex::new(MockCommandInner::default())), + } + } + + pub(crate) fn reset(&self) -> mlua::Result<()> { + let mut inner = self.lock()?; + inner.mocks.clear(); + inner.calls.clear(); + Ok(()) + } + + pub(crate) fn push_mock(&self, pattern: String, result: MockCommandResult) -> mlua::Result<()> { + let mut inner = self.lock()?; + inner.mocks.push(MockCommand { pattern, result }); + Ok(()) + } + + pub(crate) fn calls(&self) -> mlua::Result> { + let inner = self.lock()?; + Ok(inner.calls.clone()) + } + + pub(crate) fn execute( + &self, + rendered: String, + program: String, + args: Vec, + stdin: String, + ) -> mlua::Result { + let mut inner = self.lock()?; + let index = inner + .mocks + .iter() + .position(|mock| { + rendered.starts_with(&mock.pattern) || rendered.contains(&mock.pattern) + }) + .ok_or_else(|| { + mlua::Error::external(format!("unmocked external command: {rendered}")) + })?; + let mock = inner.mocks.remove(index); + inner.calls.push(MockCommandCall { + rendered, + program, + args, + stdin, + stdout: mock.result.stdout.clone(), + stderr: mock.result.stderr.clone(), + exit_code: mock.result.exit_code, + }); + Ok(mock.result) + } + + fn lock(&self) -> mlua::Result> { + self.inner + .lock() + .map_err(|_| mlua::Error::external("mock command state lock is poisoned")) + } +} + +pub(crate) fn format_command(program: &str, args: &[String]) -> String { + std::iter::once(program.to_string()) + .chain(args.iter().map(|arg| shell_quote(arg))) + .collect::>() + .join(" ") +} + +fn shell_quote(value: &str) -> String { + if value + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'_' | b'-' | b'.' | b'/' | b':' | b'=')) + { + value.to_string() + } else { + format!("'{}'", value.replace('\'', "'\"'\"'")) + } +} diff --git a/crates/fkst-framework/src/main.rs b/crates/fkst-framework/src/main.rs index 22c2d2b..8df6170 100644 --- a/crates/fkst-framework/src/main.rs +++ b/crates/fkst-framework/src/main.rs @@ -18,6 +18,7 @@ use serde_json::Value as JsonValue; use std::path::PathBuf; mod config_registry; +mod external_command; mod host_conformance; mod mlua_init; mod path_resolver; diff --git a/crates/fkst-framework/src/mlua_init.rs b/crates/fkst-framework/src/mlua_init.rs index c9a9e28..1b39e1d 100644 --- a/crates/fkst-framework/src/mlua_init.rs +++ b/crates/fkst-framework/src/mlua_init.rs @@ -6,6 +6,7 @@ use serde_json::Value as JsonValue; use std::path::Path; use crate::config_registry::ConfigContext; +use crate::external_command::MockCommandState; use crate::path_resolver::{package_root_path, NameResolver}; use crate::raise::RaiseBuffer; @@ -35,6 +36,27 @@ pub fn register_framework_sdk( Ok(()) } +pub(crate) fn register_framework_sdk_with_runner( + lua: &Lua, + raise_buf: RaiseBuffer, + host_root: &Path, + resolver: NameResolver, + owner_namespace: String, + runner: Option, +) -> mlua::Result<()> { + let config = ConfigContext::from_host_root(host_root).map_err(mlua::Error::external)?; + crate::sdk_log::register(lua)?; + crate::sdk_basic::register_with_runner(lua, runner.clone())?; + crate::sdk_fs::register(lua)?; + crate::sdk_json::register(lua)?; + crate::sdk_git::register_with_runner(lua, host_root, config.clone(), runner.clone())?; + crate::sdk_mark::register(lua, host_root)?; + crate::sdk_cache::register(lua, host_root)?; + crate::sdk_codex::register_with_runner(lua, host_root, config, runner)?; + crate::raise::register(lua, raise_buf, resolver, owner_namespace)?; + Ok(()) +} + /// Convert serde_json::Value to mlua::Value via LuaSerdeExt. pub fn json_to_lua(lua: &Lua, v: &JsonValue) -> mlua::Result { lua.to_value(v) diff --git a/crates/fkst-framework/src/sdk_basic.rs b/crates/fkst-framework/src/sdk_basic.rs index 7d6e22b..5a72ecd 100644 --- a/crates/fkst-framework/src/sdk_basic.rs +++ b/crates/fkst-framework/src/sdk_basic.rs @@ -9,6 +9,8 @@ use std::process::{Command, Stdio}; use std::thread::JoinHandle; use std::time::{Duration, Instant}; +use crate::external_command::MockCommandState; + struct ExecOptions { cmd: String, cwd: Option, @@ -25,6 +27,10 @@ struct ExecResult { // Lua SDK registration and self-test match the fixed CLAUDE.md surface exactly; human notification, if needed, is represented through existing git/fs/log facts rather than a new SDK function. pub fn register(lua: &Lua) -> Result<()> { + register_with_runner(lua, None) +} + +pub(crate) fn register_with_runner(lua: &Lua, runner: Option) -> Result<()> { lua.globals().set( "now", lua.create_function(|_, ()| { @@ -38,9 +44,9 @@ pub fn register(lua: &Lua) -> Result<()> { lua.globals().set( "exec_sync", - lua.create_function(|lua, arg: Value| { + lua.create_function(move |lua, arg: Value| { let opts = parse_exec_options(arg)?; - let out = run_exec_sync(opts)?; + let out = run_exec_sync(opts, runner.as_ref())?; let t = lua.create_table()?; t.set("stdout", out.stdout)?; t.set("stderr", out.stderr)?; @@ -123,7 +129,22 @@ fn kill_process_group(child_pid: u32) { #[cfg(not(unix))] fn kill_process_group(_child_pid: u32) {} -fn run_exec_sync(opts: ExecOptions) -> Result { +fn run_exec_sync(opts: ExecOptions, runner: Option<&MockCommandState>) -> Result { + if let Some(runner) = runner { + let result = runner.execute( + opts.cmd.clone(), + "/bin/sh".to_string(), + vec!["-c".to_string(), opts.cmd], + String::new(), + )?; + return Ok(ExecResult { + stdout: result.stdout, + stderr: result.stderr, + exit_code: result.exit_code, + timed_out: None, + }); + } + match opts.timeout { Some(timeout) => run_exec_sync_with_timeout(&opts, timeout), None => { diff --git a/crates/fkst-framework/src/sdk_codex.rs b/crates/fkst-framework/src/sdk_codex.rs index 65429ce..3902477 100644 --- a/crates/fkst-framework/src/sdk_codex.rs +++ b/crates/fkst-framework/src/sdk_codex.rs @@ -22,6 +22,7 @@ use std::thread::JoinHandle; use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; use crate::config_registry::{ConfigContext, ConfigKey}; +use crate::external_command::{format_command, MockCommandState}; use crate::runtime_context; pub(crate) const CODEX_PERMIT_SLOTS_ENV: &str = "FKST_CODEX_PERMIT_SLOTS"; @@ -140,6 +141,18 @@ impl CodexResult { } } + fn mock_error(err: mlua::Error) -> Self { + let message = err.to_string(); + Self { + stdout: String::new(), + stderr: message.clone(), + exit_code: -1, + log_path: String::new(), + error_kind: Some("mock".to_string()), + error: Some(message), + } + } + fn into_lua_table(self, lua: &Lua) -> Result { let t = result_table(lua, self.stdout, self.stderr, self.exit_code, self.log_path)?; if let Some(kind) = self.error_kind { @@ -168,29 +181,46 @@ fn result_table( } pub fn register(lua: &Lua, host_root: &Path, config: ConfigContext) -> Result<()> { + register_with_runner(lua, host_root, config, None) +} + +pub(crate) fn register_with_runner( + lua: &Lua, + host_root: &Path, + config: ConfigContext, + runner: Option, +) -> Result<()> { let owner_id = NEXT_PIPELINE_OWNER_ID.fetch_add(1, Ordering::Relaxed); let next_task_id = Arc::new(AtomicU64::new(1)); let host_root = Arc::new(host_root.to_path_buf()); let config = Arc::new(config); + let runner = Arc::new(runner); lua.globals().set("spawn_codex_sync", { let host_root = Arc::clone(&host_root); let config = Arc::clone(&config); + let runner = Arc::clone(&runner); lua.create_function(move |lua, opts: Table| { let request = codex_request_from_opts(opts); - run_codex_request(request, &host_root, &config).into_lua_table(lua) + run_codex_request(request, &host_root, &config, runner.as_ref().as_ref())? + .into_lua_table(lua) })? })?; lua.globals().set("spawn_codex", { let next_task_id = Arc::clone(&next_task_id); let host_root = Arc::clone(&host_root); let config = Arc::clone(&config); + let runner = Arc::clone(&runner); lua.create_function(move |_, opts: Table| { let request = codex_request_from_opts(opts); let task_id = next_task_id.fetch_add(1, Ordering::Relaxed); let host_root = Arc::clone(&host_root); let config = Arc::clone(&config); - let join = std::thread::spawn(move || run_codex_request(request, &host_root, &config)); + let runner = Arc::clone(&runner); + let join = std::thread::spawn(move || { + run_codex_request(request, &host_root, &config, runner.as_ref().as_ref()) + .unwrap_or_else(CodexResult::mock_error) + }); Ok(CodexTaskHandle { owner_id, task_id, @@ -292,6 +322,14 @@ fn await_all(lua: &Lua, handles: Table, owner_id: u64) -> Result
{ String::new(), ), }; + if result.error_kind.as_deref() == Some("mock") { + return Err(mlua::Error::external( + result + .error + .clone() + .unwrap_or_else(|| "mock command failed".to_string()), + )); + } results.set(index + 1, result.into_lua_table(lua)?)?; } if let Some(message) = invalid { @@ -305,7 +343,12 @@ fn run_codex_request( request: CodexRequest, host_root: &Path, config: &ConfigContext, -) -> CodexResult { + runner: Option<&MockCommandState>, +) -> Result { + if let Some(runner) = runner { + return run_mocked_codex_request(request, runner); + } + if let Err(err) = ensure_pool_with_context(host_root, config) { let message = format!("codex permit pool failed: {err}"); write_codex_log( @@ -316,14 +359,14 @@ fn run_codex_request( &command_line_for_request(&request), request.stall_window_seconds, ); - return CodexResult::failure( + return Ok(CodexResult::failure( "permit", message, String::new(), String::new(), -1, request.log_path.to_string_lossy().into_owned(), - ); + )); } let _permit = match acquire_permit_with_context(host_root, config) { Ok(permit) => permit, @@ -337,18 +380,46 @@ fn run_codex_request( &command_line_for_request(&request), request.stall_window_seconds, ); - return CodexResult::failure( + return Ok(CodexResult::failure( "permit", message, String::new(), String::new(), -1, request.log_path.to_string_lossy().into_owned(), - ); + )); } }; - run_codex_request_with_permit(request) + Ok(run_codex_request_with_permit(request)) +} + +fn run_mocked_codex_request( + request: CodexRequest, + runner: &MockCommandState, +) -> Result { + let args = command_args_for_request(&request); + let cmd_line = format_command("codex", &args); + let result = runner.execute( + cmd_line.clone(), + "codex".to_string(), + args, + request.prompt.clone(), + )?; + write_codex_log( + &request.log_path, + &result.stdout, + &result.stderr, + result.exit_code, + &cmd_line, + request.stall_window_seconds, + ); + Ok(CodexResult::success( + result.stdout, + result.stderr, + result.exit_code, + request.log_path.to_string_lossy().into_owned(), + )) } fn run_codex_request_with_permit(request: CodexRequest) -> CodexResult { @@ -832,24 +903,6 @@ fn civil_from_days(days_since_epoch: i64) -> (i32, u32, u32) { (year as i32, month as u32, day as u32) } -fn format_command(program: &str, args: &[String]) -> String { - std::iter::once(program.to_string()) - .chain(args.iter().map(|arg| shell_quote(arg))) - .collect::>() - .join(" ") -} - -fn shell_quote(value: &str) -> String { - if value - .bytes() - .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'_' | b'-' | b'.' | b'/' | b':' | b'=')) - { - value.to_string() - } else { - format!("'{}'", value.replace('\'', "'\"'\"'")) - } -} - // crate-internal visibility keeps permit setup testable after extraction. #[cfg(test)] pub(crate) fn ensure_pool() -> mlua::Result<()> { diff --git a/crates/fkst-framework/src/sdk_git.rs b/crates/fkst-framework/src/sdk_git.rs index 115e8ae..0e01690 100644 --- a/crates/fkst-framework/src/sdk_git.rs +++ b/crates/fkst-framework/src/sdk_git.rs @@ -11,20 +11,30 @@ use mlua::{Function, Lua, Result}; use nix::fcntl::{flock, FlockArg}; use std::os::fd::AsRawFd; use std::path::{Path, PathBuf}; -use std::process::Command; +use std::process::{Command, Output}; use std::time::{SystemTime, UNIX_EPOCH}; use crate::config_registry::{ConfigContext, ConfigKey}; +use crate::external_command::{format_command, MockCommandState}; use crate::runtime_context; pub fn register(lua: &Lua, host_root: &Path, config: ConfigContext) -> Result<()> { + register_with_runner(lua, host_root, config, None) +} + +pub(crate) fn register_with_runner( + lua: &Lua, + host_root: &Path, + config: ConfigContext, + runner: Option, +) -> Result<()> { let host_root = host_root.to_path_buf(); register_with_lock(lua, host_root.clone())?; - register_setup_worktree(lua, host_root.clone(), config)?; - register_git_log_count(lua, host_root.clone())?; - register_git_log_grep(lua, host_root.clone())?; - register_count_worktrees(lua, host_root.clone())?; - register_list_orphan_worktrees(lua, host_root)?; + register_setup_worktree(lua, host_root.clone(), config, runner.clone())?; + register_git_log_count(lua, host_root.clone(), runner.clone())?; + register_git_log_grep(lua, host_root.clone(), runner.clone())?; + register_count_worktrees(lua, host_root.clone(), runner.clone())?; + register_list_orphan_worktrees(lua, host_root, runner)?; Ok(()) } @@ -75,7 +85,12 @@ fn register_with_lock(lua: &Lua, host_root: PathBuf) -> Result<()> { Ok(()) } -fn register_setup_worktree(lua: &Lua, host_root: PathBuf, config: ConfigContext) -> Result<()> { +fn register_setup_worktree( + lua: &Lua, + host_root: PathBuf, + config: ConfigContext, + runner: Option, +) -> Result<()> { lua.globals().set( "setup_worktree", lua.create_function(move |_, prefix: String| { @@ -86,7 +101,7 @@ fn register_setup_worktree(lua: &Lua, host_root: PathBuf, config: ConfigContext) let worktrees = layout.runtime_dir(RuntimeKind::Worktrees); let path = worktrees.join(format!("{}-{}", prefix, ulid)); let path = path.to_string_lossy().into_owned(); - let parent_ref = current_parent_ref(&host_root)?; + let parent_ref = current_parent_ref_with_runner(&host_root, runner.as_ref())?; let parent_name = branch_slug(&parent_ref); let branch = format!( "{}-{}-{}-{}{}{}", @@ -101,10 +116,11 @@ fn register_setup_worktree(lua: &Lua, host_root: PathBuf, config: ConfigContext) // worktrees live below RuntimeLayout worktrees dir. std::fs::create_dir_all(worktrees).map_err(mlua::Error::external)?; - let out = git_command(&host_root) - .args(["worktree", "add", "-b", &branch, &path, &parent_ref]) - .output() - .map_err(mlua::Error::external)?; + let out = run_git_command( + &host_root, + ["worktree", "add", "-b", &branch, &path, &parent_ref], + runner.as_ref(), + )?; if !out.status.success() { let err = String::from_utf8_lossy(&out.stderr); @@ -152,11 +168,11 @@ fn validate_branch_fragment(name: &str, value: &str) -> Result<()> { Ok(()) } -fn current_parent_ref(host_root: &Path) -> Result { - let branch = git_command(host_root) - .args(["rev-parse", "--abbrev-ref", "HEAD"]) - .output() - .map_err(mlua::Error::external)?; +fn current_parent_ref_with_runner( + host_root: &Path, + runner: Option<&MockCommandState>, +) -> Result { + let branch = run_git_command(host_root, ["rev-parse", "--abbrev-ref", "HEAD"], runner)?; if !branch.status.success() { return Err(read_failure("git-current-branch-failed", &branch.stderr)); } @@ -165,10 +181,7 @@ fn current_parent_ref(host_root: &Path) -> Result { return Ok(branch); } - let sha = git_command(host_root) - .args(["rev-parse", "--short=12", "HEAD"]) - .output() - .map_err(mlua::Error::external)?; + let sha = run_git_command(host_root, ["rev-parse", "--short=12", "HEAD"], runner)?; if !sha.status.success() { return Err(read_failure("git-current-sha-failed", &sha.stderr)); } @@ -226,19 +239,24 @@ fn civil_from_days(days_since_epoch: i64) -> (i32, u32, u32) { (year as i32, month as u32, day as u32) } -fn register_git_log_count(lua: &Lua, host_root: PathBuf) -> Result<()> { +fn register_git_log_count( + lua: &Lua, + host_root: PathBuf, + runner: Option, +) -> Result<()> { lua.globals().set( "git_log_count", lua.create_function(move |_, (grep, since): (String, String)| { - let out = git_command(&host_root) - .args([ + let out = run_git_command( + &host_root, + [ "log", &format!("--grep={}", grep), &format!("--since={}", since), "--oneline", - ]) - .output() - .map_err(mlua::Error::external)?; + ], + runner.as_ref(), + )?; if !out.status.success() { return Err(read_failure("git-log-count-failed", &out.stderr)); @@ -251,19 +269,24 @@ fn register_git_log_count(lua: &Lua, host_root: PathBuf) -> Result<()> { Ok(()) } -fn register_git_log_grep(lua: &Lua, host_root: PathBuf) -> Result<()> { +fn register_git_log_grep( + lua: &Lua, + host_root: PathBuf, + runner: Option, +) -> Result<()> { lua.globals().set( "git_log_grep", lua.create_function(move |_, (grep, since): (String, String)| { - let out = git_command(&host_root) - .args([ + let out = run_git_command( + &host_root, + [ "log", &format!("--grep={}", grep), &format!("--since={}", since), "--format=%H", - ]) - .output() - .map_err(mlua::Error::external)?; + ], + runner.as_ref(), + )?; if !out.status.success() { return Err(read_failure("git-log-grep-failed", &out.stderr)); @@ -290,14 +313,19 @@ pub(crate) fn parse_worktree_paths(stdout: &[u8]) -> Vec { .collect() } -fn register_count_worktrees(lua: &Lua, host_root: PathBuf) -> Result<()> { +fn register_count_worktrees( + lua: &Lua, + host_root: PathBuf, + runner: Option, +) -> Result<()> { lua.globals().set( "count_worktrees", lua.create_function(move |_, ()| { - let out = git_command(&host_root) - .args(["worktree", "list", "--porcelain"]) - .output() - .map_err(mlua::Error::external)?; + let out = run_git_command( + &host_root, + ["worktree", "list", "--porcelain"], + runner.as_ref(), + )?; if !out.status.success() { return Err(read_failure("git-worktree-list-failed", &out.stderr)); @@ -310,14 +338,19 @@ fn register_count_worktrees(lua: &Lua, host_root: PathBuf) -> Result<()> { Ok(()) } -fn register_list_orphan_worktrees(lua: &Lua, host_root: PathBuf) -> Result<()> { +fn register_list_orphan_worktrees( + lua: &Lua, + host_root: PathBuf, + runner: Option, +) -> Result<()> { lua.globals().set( "list_orphan_worktrees", lua.create_function(move |_, prefix: String| { - let out = git_command(&host_root) - .args(["worktree", "list", "--porcelain"]) - .output() - .map_err(mlua::Error::external)?; + let out = run_git_command( + &host_root, + ["worktree", "list", "--porcelain"], + runner.as_ref(), + )?; if !out.status.success() { return Err(read_failure("git-worktree-list-failed", &out.stderr)); @@ -348,6 +381,78 @@ fn git_command(host_root: &Path) -> Command { command } +fn run_git_command<'a>( + host_root: &Path, + args: impl IntoIterator, + runner: Option<&MockCommandState>, +) -> Result { + let mut rendered_args = vec!["-C".to_string(), host_root.to_string_lossy().into_owned()]; + rendered_args.extend(args.into_iter().map(ToOwned::to_owned)); + if let Some(runner) = runner { + let result = runner.execute( + format_command("git", &rendered_args), + "git".to_string(), + rendered_args, + String::new(), + )?; + return Ok(mock_output(result.stdout, result.stderr, result.exit_code)); + } + + git_command(host_root) + .args(rendered_args.iter().skip(2)) + .output() + .map_err(mlua::Error::external) +} + +#[cfg(unix)] +fn mock_output(stdout: String, stderr: String, exit_code: i32) -> Output { + use std::os::unix::process::ExitStatusExt; + + Output { + status: std::process::ExitStatus::from_raw(exit_code << 8), + stdout: stdout.into_bytes(), + stderr: stderr.into_bytes(), + } +} + +#[cfg(not(unix))] +fn mock_output(stdout: String, stderr: String, exit_code: i32) -> Output { + Output { + status: portable_exit_status(exit_code), + stdout: stdout.into_bytes(), + stderr: stderr.into_bytes(), + } +} + +#[cfg(not(unix))] +fn portable_exit_status(exit_code: i32) -> std::process::ExitStatus { + let code = exit_code.clamp(0, 255).to_string(); + std::process::Command::new(platform_shell()) + .args(platform_shell_exit_args(code)) + .status() + .expect("platform shell must produce mock command exit status") +} + +#[cfg(all(not(unix), windows))] +fn platform_shell() -> &'static str { + "cmd" +} + +#[cfg(all(not(unix), windows))] +fn platform_shell_exit_args(code: String) -> Vec { + vec!["/C".to_string(), "exit".to_string(), code] +} + +#[cfg(all(not(unix), not(windows)))] +fn platform_shell() -> &'static str { + "sh" +} + +#[cfg(all(not(unix), not(windows)))] +fn platform_shell_exit_args(code: String) -> Vec { + vec!["-c".to_string(), format!("exit {code}")] +} + fn runtime_dir_abs(layout: &RuntimeLayout, kind: RuntimeKind) -> std::path::PathBuf { let path = layout.runtime_dir(kind); match std::fs::canonicalize(&path) { diff --git a/crates/fkst-framework/src/test_runner.rs b/crates/fkst-framework/src/test_runner.rs index e9e085e..cb30d11 100644 --- a/crates/fkst-framework/src/test_runner.rs +++ b/crates/fkst-framework/src/test_runner.rs @@ -3,6 +3,7 @@ use mlua::{Function, Lua, LuaSerdeExt, Table, Value}; use std::collections::BTreeMap; use std::path::{Path, PathBuf}; +use crate::external_command::{MockCommandResult, MockCommandState}; use crate::path_resolver::PackageRoots; use crate::raise::RaiseBuffer; @@ -13,15 +14,17 @@ pub(crate) fn run_tests(roots: PackageRoots) -> Result { for file in files { let relpath = display_path(&file.path, &file.owner_root); + let mock_commands = MockCommandState::new(); let lua = crate::mlua_init::new_lua(); crate::mlua_init::set_package_roots_path(&lua, [file.owner_root.as_path()]) .with_context(|| format!("set package.path for tests in {}", relpath))?; - crate::mlua_init::register_framework_sdk( + crate::mlua_init::register_framework_sdk_with_runner( &lua, RaiseBuffer::new(), roots.host_root(), roots.name_resolver(), file.owner_namespace.clone(), + Some(mock_commands.clone()), ) .with_context(|| format!("register SDK for {}", relpath))?; register_test_sdk( @@ -29,12 +32,16 @@ pub(crate) fn run_tests(roots: PackageRoots) -> Result { roots.clone(), file.owner_root.clone(), file.owner_namespace.clone(), + mock_commands.clone(), ) .with_context(|| format!("register fkst.test for {}", relpath))?; match load_test_table(&lua, &file.path) { Ok(tests) => { for (name, func) in tests { + mock_commands + .reset() + .with_context(|| format!("reset mock commands for {relpath}::{name}"))?; match func.call::<()>(()) { Ok(()) => { println!("PASS {relpath}::{name}"); @@ -172,6 +179,7 @@ fn register_test_sdk( roots: PackageRoots, owner_root: PathBuf, owner_namespace: String, + mock_commands: MockCommandState, ) -> mlua::Result<()> { let globals = lua.globals(); let fkst = match globals.get::("fkst")? { @@ -242,8 +250,46 @@ fn register_test_sdk( )) })?, )?; - test.set( - "run_department", + test.set("mock_command", { + let mock_commands = mock_commands.clone(); + lua.create_function(move |_, (pattern, result): (String, Table)| { + let stdout = result.get::>("stdout")?.unwrap_or_default(); + let stderr = result.get::>("stderr")?.unwrap_or_default(); + let exit_code = result.get::>("exit_code")?.unwrap_or(0); + mock_commands.push_mock( + pattern, + MockCommandResult { + stdout, + stderr, + exit_code, + }, + ) + })? + })?; + test.set("command_calls", { + let mock_commands = mock_commands.clone(); + lua.create_function(move |lua, ()| { + let calls = lua.create_table()?; + for (idx, call) in mock_commands.calls()?.into_iter().enumerate() { + let entry = lua.create_table()?; + entry.set("rendered", call.rendered)?; + entry.set("program", call.program)?; + let args = lua.create_table()?; + for (arg_idx, arg) in call.args.into_iter().enumerate() { + args.set(arg_idx + 1, arg)?; + } + entry.set("args", args)?; + entry.set("stdin", call.stdin)?; + entry.set("stdout", call.stdout)?; + entry.set("stderr", call.stderr)?; + entry.set("exit_code", call.exit_code)?; + calls.set(idx + 1, entry)?; + } + Ok(calls) + })? + })?; + test.set("run_department", { + let mock_commands = mock_commands.clone(); lua.create_function( move |lua, (path, event, opts): (String, Value, Option
)| { run_department( @@ -251,13 +297,14 @@ fn register_test_sdk( &roots, &owner_root, &owner_namespace, + mock_commands.clone(), path, event, opts, ) }, - )?, - )?; + )? + })?; fkst.set("test", test)?; globals.set("fkst", fkst)?; Ok(()) @@ -268,6 +315,7 @@ fn run_department( roots: &PackageRoots, owner_root: &Path, owner_namespace: &str, + mock_commands: MockCommandState, path: String, event: Value, opts: Option
, @@ -279,12 +327,13 @@ fn run_department( let dept_lua = crate::mlua_init::new_lua(); let raise_buf = RaiseBuffer::new(); - crate::mlua_init::register_framework_sdk( + crate::mlua_init::register_framework_sdk_with_runner( &dept_lua, raise_buf.clone(), roots.host_root(), roots.name_resolver(), owner_namespace.to_string(), + Some(mock_commands), )?; let require_roots = roots.require_roots_for_owner(owner_root); diff --git a/crates/fkst-framework/tests/sdk_codex.rs b/crates/fkst-framework/tests/sdk_codex.rs index d64a9bf..ca749e6 100644 --- a/crates/fkst-framework/tests/sdk_codex.rs +++ b/crates/fkst-framework/tests/sdk_codex.rs @@ -2,6 +2,8 @@ #[path = "../src/config_registry.rs"] mod config_registry; +#[path = "../src/external_command.rs"] +mod external_command; #[path = "../src/runtime_context.rs"] mod runtime_context; #[path = "../src/sdk_codex.rs"] diff --git a/crates/fkst-framework/tests/sdk_git.rs b/crates/fkst-framework/tests/sdk_git.rs index 7b11955..3881e24 100644 --- a/crates/fkst-framework/tests/sdk_git.rs +++ b/crates/fkst-framework/tests/sdk_git.rs @@ -2,6 +2,8 @@ #[path = "../src/config_registry.rs"] mod config_registry; +#[path = "../src/external_command.rs"] +mod external_command; #[path = "../src/runtime_context.rs"] mod runtime_context; #[path = "../src/sdk_git.rs"] diff --git a/crates/fkst-framework/tests/test_runner_cli.rs b/crates/fkst-framework/tests/test_runner_cli.rs index 1394785..8b73041 100644 --- a/crates/fkst-framework/tests/test_runner_cli.rs +++ b/crates/fkst-framework/tests/test_runner_cli.rs @@ -320,6 +320,208 @@ return { assert!(out.contains("2 passed, 1 failed"), "stdout: {out}"); } +#[test] +fn test_runner_mocks_external_commands_fail_closed_and_isolates_tests() { + let host = tempfile::Builder::new().prefix("repo").tempdir().unwrap(); + fs::write( + host.path().join("fkst.env"), + "FKST_CANDIDATE_PREFIX=test-rc\nFKST_CANDIDATE_FROM_SEP=__base__\n", + ) + .unwrap(); + fs::create_dir_all(host.path().join("departments/probe")).unwrap(); + fs::write( + host.path().join("departments/probe/main.lua"), + r#" +function pipeline(event) + local result = exec_sync(event.payload.cmd) + raise("seen", { stdout = result.stdout, exit_code = result.exit_code }) +end +"#, + ) + .unwrap(); + fs::create_dir_all(host.path().join("tests")).unwrap(); + fs::write( + host.path().join("tests/mock_command_test.lua"), + r#" +local t = fkst.test + +return { + test_01_exec_sync_uses_mock_and_records_call = function() + t.mock_command("gh issue list", { stdout = "[{\"number\":7}]\n", exit_code = 0 }) + local result = exec_sync("gh issue list --json number") + t.eq(result.stdout, "[{\"number\":7}]\n") + t.eq(result.exit_code, 0) + local calls = t.command_calls() + t.eq(#calls, 1) + t.eq(calls[1].rendered, "gh issue list --json number") + t.eq(calls[1].program, "/bin/sh") + t.eq(calls[1].args[1], "-c") + t.eq(calls[1].args[2], "gh issue list --json number") + end, + + test_02_codex_sync_uses_mock_and_records_prompt = function() + t.mock_command("codex exec", { stdout = "draft", exit_code = 0 }) + local result = spawn_codex_sync({ prompt = "write draft" }) + t.eq(result.stdout, "draft") + t.eq(result.stderr, "") + t.eq(result.exit_code, 0) + local calls = t.command_calls() + t.eq(#calls, 1) + t.eq(calls[1].program, "codex") + t.eq(calls[1].stdin, "write draft") + t.is_true(string.find(calls[1].rendered, "codex exec", 1, true) ~= nil) + t.eq(calls[1].args[#calls[1].args], "-") + end, + + test_03_codex_sync_nonzero_and_empty_stdout_pass_through = function() + t.mock_command("codex exec", { stderr = "nope", exit_code = 12 }) + local result = spawn_codex_sync({ prompt = "fail draft" }) + t.eq(result.stdout, "") + t.eq(result.stderr, "nope") + t.eq(result.exit_code, 12) + end, + + test_04_git_log_count_uses_mock_stdout = function() + t.mock_command("git -C", { stdout = "a\nb\nc\n" }) + local count = git_log_count("topic", "1970-01-01T00:00:00Z") + t.eq(count, 3) + local calls = t.command_calls() + t.is_true(string.find(calls[1].rendered, "git -C", 1, true) ~= nil) + t.is_true(string.find(calls[1].rendered, "log", 1, true) ~= nil) + end, + + test_05_git_read_primitives_parse_mocked_stdout = function() + local worktree_stdout = table.concat({ + "worktree /repo", + "HEAD aaaaaaaaaaaa", + "", + "worktree /repo/.fkst/runtime/worktrees/probe-1", + "HEAD bbbbbbbbbbbb", + "", + "worktree /repo/.fkst/runtime/worktrees/probe-2", + "HEAD cccccccccccc", + "", + }, "\n") + t.mock_command("git -C", { stdout = worktree_stdout }) + t.eq(count_worktrees(), 2) + + t.mock_command("git -C", { stdout = "abc123\n\n def456 \n" }) + local shas = git_log_grep("topic", "1970-01-01T00:00:00Z") + t.eq(#shas, 2) + t.eq(shas[1], "abc123") + t.eq(shas[2], "def456") + + local calls = t.command_calls() + t.eq(#calls, 2) + t.is_true(string.find(calls[1].rendered, "worktree list --porcelain", 1, true) ~= nil) + t.is_true(string.find(calls[2].rendered, "--format=%H", 1, true) ~= nil) + end, + + test_06_mock_commands_are_fifo_and_single_use = function() + t.mock_command("git -C", { stdout = "first\n" }) + t.mock_command("git -C", { stdout = "second\nthird\n" }) + t.eq(git_log_count("topic", "now"), 1) + t.eq(git_log_count("topic", "now"), 2) + + local ok, err = pcall(function() + git_log_count("topic", "now") + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: git -C", 1, true) ~= nil) + end, + + test_07_setup_worktree_fails_closed_when_later_git_call_is_unmocked = function() + t.mock_command("git -C", { stdout = "main\n" }) + local ok, err = pcall(function() + setup_worktree("mocked") + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: git -C", 1, true) ~= nil) + + local calls = t.command_calls() + t.eq(#calls, 1) + t.is_true(string.find(calls[1].rendered, "rev-parse --abbrev-ref HEAD", 1, true) ~= nil) + end, + + test_08_unmocked_exec_sync_fails_closed = function() + local ok, err = pcall(function() + exec_sync("printf should-not-run") + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: printf should-not-run", 1, true) ~= nil) + end, + + test_09_unmocked_codex_sync_fails_closed = function() + local ok, err = pcall(function() + spawn_codex_sync({ prompt = "unmocked" }) + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: codex exec", 1, true) ~= nil) + end, + + test_10_unmocked_git_fails_closed = function() + local ok, err = pcall(function() + git_log_count("topic", "now") + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: git -C", 1, true) ~= nil) + end, + + test_11_spawn_codex_await_all_uses_mock = function() + t.mock_command("codex exec", { stdout = "async draft", exit_code = 0 }) + local handle = spawn_codex({ prompt = "async prompt" }) + local results = await_all({ handle }) + t.eq(results[1].stdout, "async draft") + t.eq(results[1].stderr, "") + t.eq(results[1].exit_code, 0) + local calls = t.command_calls() + t.eq(calls[1].stdin, "async prompt") + end, + + test_12_unmocked_spawn_codex_fails_on_await = function() + local handle = spawn_codex({ prompt = "async unmocked" }) + local ok, err = pcall(function() + await_all({ handle }) + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: codex exec", 1, true) ~= nil) + end, + + test_13_run_department_shares_mock_state = function() + t.mock_command("gh issue list", { stdout = "dept\n", exit_code = 0 }) + local result = t.run_department("departments/probe/main.lua", { payload = { cmd = "gh issue list" } }) + t.eq(result.exit_code, 0) + t.eq(result.raises[1].payload.stdout, "dept\n") + local calls = t.command_calls() + t.eq(#calls, 1) + t.eq(calls[1].rendered, "gh issue list") + end, + + test_14_per_test_isolation_clears_prior_mocks_and_calls = function() + t.eq(#t.command_calls(), 0) + local ok, err = pcall(function() + exec_sync("gh issue list --json number") + end) + t.eq(ok, false) + t.is_true(string.find(tostring(err), "unmocked external command: gh issue list --json number", 1, true) ~= nil) + end, +} +"#, + ) + .unwrap(); + + let output = run_lua_tests(host.path(), host.path()); + + assert!( + output.status.success(), + "stdout: {}\nstderr: {}", + stdout(&output), + stderr(&output) + ); + let out = stdout(&output); + assert!(out.contains("14 passed, 0 failed"), "stdout: {out}"); +} + #[test] fn test_surface_does_not_leak_to_production_run() { let host = tempfile::Builder::new().prefix("repo").tempdir().unwrap(); diff --git a/docs/architecture.md b/docs/architecture.md index 0be7e14..1410cd1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -94,7 +94,9 @@ fkst-framework --self-test runner 不全树递归，不扫描 `raisers/` 或 `fkst/`。每个测试文件在独立 Lua state 中执行，先注册 production SDK 以便测试可 `require` package 模块和调用固定 SDK，再注册 test-mode `fkst.test` 表。测试文件必须返回 table；runner 只执行排序后的 `test_*` key。单个测试失败后继续执行同文件其余测试和后续文件，最后输出 `N passed, M failed`；`M > 0` 时退出码非 0。 -`fkst.test` 包含 `eq(actual, expected[, msg])`、`is_true(value[, msg])`、`raises(fn[, msg])`、`is_nil(value[, msg])` 四个断言，以及 test-mode-only `run_department(path, event[, opts])`。`run_department` 用 fresh Lua state 注册 production SDK 和独立 `RaiseBuffer`，再通过正常 department runner 注入 `event`；它返回 `{ exit_code = int, raises = { { queue = string, payload = table }, ... } }`。每个测试文件按所属 graph root 隔离执行；相对 `path` 按该测试文件所属的 owner package root 解析，运行期 `package.path` 也只指向该 owner root。绝对 `path` 仍按绝对路径处理。`opts.cwd`、`opts.env`、`opts.path_prepend` 只作用于该次执行并随后恢复。它是最小 Lua 单测工具，不提供 describe/it、hook、fixture、mock、stub 或测试框架 DSL。除非明确验证真实 CLI 路径，Lua 单测不应调用 codex。 +`fkst.test` 包含 `eq(actual, expected[, msg])`、`is_true(value[, msg])`、`raises(fn[, msg])`、`is_nil(value[, msg])` 四个断言，以及 test-mode-only `run_department(path, event[, opts])`。`run_department` 用 fresh Lua state 注册 production SDK 和独立 `RaiseBuffer`，再通过正常 department runner 注入 `event`；它返回 `{ exit_code = int, raises = { { queue = string, payload = table }, ... } }`。每个测试文件按所属 graph root 隔离执行；相对 `path` 按该测试文件所属的 owner package root 解析，运行期 `package.path` 也只指向该 owner root。绝对 `path` 仍按绝对路径处理。`opts.cwd`、`opts.env`、`opts.path_prepend` 只作用于该次执行并随后恢复。 + +`fkst.test.mock_command(pattern, result)` 与 `fkst.test.command_calls()` 只在 test mode 注册。mock runner 劫持 `exec_sync`、codex SDK 与 git SDK 的外部命令调用；渲染命令行按前缀或子串匹配，mock 按注册顺序一次性消费，未 mock 的外部命令 fail closed 且不启动真实进程。`command_calls()` 返回每次调用的 rendered、program、args、stdin、stdout、stderr 与 exit_code。`run_department` 创建的 fresh Lua state 与测试文件共享同一个 mock state；每个 test function 开始前清空 mocks 与 calls。`setup_worktree` 在 test mode 也经过同一 git mock runner，但 mock 不模拟 worktree filesystem 副作用。 ## 4. Package Root 与 Host Root diff --git a/examples/codex-package/README.md b/examples/codex-package/README.md index 643567b..84feb6a 100644 --- a/examples/codex-package/README.md +++ b/examples/codex-package/README.md @@ -69,7 +69,7 @@ Lua 单元测试用 `fkst-framework test` 执行。runner 只扫描 `departments --package-root "$tmp_host" ``` -`fkst.test` 当前只提供 `eq`、`is_true`、`raises`、`is_nil` 四个断言。它是最小 test-mode 工具，不提供 fixture、mock、hook 或测试框架 DSL；本 fixture 的 Lua 单测只测纯字符串逻辑，不调用 codex。 +`fkst.test` 提供 `eq`、`is_true`、`raises`、`is_nil` 四个断言，并提供 `mock_command(pattern, result)` 与 `command_calls()` 劫持 test mode 外部命令调用。本 fixture 的 Lua 单测只测纯字符串逻辑，不调用 codex。 如果要手动验证 codex 调用路径，可以放一个 fake `codex` 到 `PATH` 前面： diff --git a/examples/minimal-package/README.md b/examples/minimal-package/README.md index 3a20612..0386fdf 100644 --- a/examples/minimal-package/README.md +++ b/examples/minimal-package/README.md @@ -58,7 +58,7 @@ target/debug/fkst-framework test \ `run producer` 的 stdout 应包含 `RAISED:`，解码后 queue 是 `example_event`。`run consumer` 的 stderr 应包含 `consumer received Event{queue=example_event`。这两个命令都是单 pipeline 注入，不经过 supervise 路由。 -Lua 单元测试由 `fkst-framework test` 执行。runner 只扫描 `departments/*/*_test.lua` 和 `tests/*_test.lua`，不全树递归，也不扫描 `raisers/` 或 `fkst/`。`fkst.test` 只在 test-mode 注册，断言包含 `eq`、`is_true`、`raises`、`is_nil`；`run_department(path, event[, opts])` 可用 fresh Lua state 运行一个 department entrypoint 并返回 captured raises。它不是 production SDK surface，也不是 mock / fixture 框架。 +Lua 单元测试由 `fkst-framework test` 执行。runner 只扫描 `departments/*/*_test.lua` 和 `tests/*_test.lua`，不全树递归，也不扫描 `raisers/` 或 `fkst/`。`fkst.test` 只在 test-mode 注册，断言包含 `eq`、`is_true`、`raises`、`is_nil`；`run_department(path, event[, opts])` 可用 fresh Lua state 运行一个 department entrypoint 并返回 captured raises。test-mode 还提供 `mock_command(pattern, result)` 与 `command_calls()`，用于劫持 `exec_sync`、codex SDK 与 git SDK 的外部命令调用。它不是 production SDK surface。 可以手动运行 supervise 观察真实路由，运行后用 `Ctrl-C` 停止；它不是 example 测试依赖：