Skip to content

CHANGELOG.md

Latest commit

 

History

History
222 lines (189 loc) · 10.7 KB

CHANGELOG.md

File metadata and controls

222 lines (189 loc) · 10.7 KB
Raw

Changelog

All notable changes to this project are documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Commit messages follow the Angular convention defined in ADR-0024.

Unreleased

Added

  • TmpFile capture mode for subprocess.spawn (ADR-0054 revision 2) — persists stdout/stderr to .substrate-subprocess-stream-<job_id>.<stream> via atomic rename of .tmp.<uuid7> transit file; coexists with live notifications/progress channel.

  • subprocess.tmp_root TOML config key (defaults to first policy.roots entry) — controls the directory where TmpFile capture-mode stream transit and final files reside (ADR-0017 revision 2).

  • SubprocessResult.stdout_tmp_path / stderr_tmp_path Option<PathBuf> fields — absolute paths to persisted final stream files; set only when capture_kind = "tmp_file" and the job terminates with Succeeded.

  • #SubprocessResult CUE definition added to docs/arch/schemas/subprocess.cue formalising the terminal output shape including tmp path fields.

  • subprocess bounded context (ADR-0052 supersedes ADR-0044) with tools subprocess.spawn, subprocess.list, subprocess.cancel, subprocess.result, and subprocess.signal behind Cargo feature subprocess (default-OFF); includes CUE schema, Rego invariants, 12 Gherkin features, domain README, and Structurizr DSL container entry.

  • ADR-0050: system resource monitoring (sys.mem, sys.cpu).

  • ADR-0051: per-process resource stats (proc.stats, proc.top).

  • ADR-0053: process lifecycle cascade contract (setsid + PR_SET_PDEATHSIG on Linux + macOS watchdog pipe for orphan prevention).

  • ADR-0054: subprocess stdout/stderr stream multiplex via notifications/progress chunked payload (StreamChunk value object).

  • ADR-0055: orphan reaper on startup removes stale .tmp.<uuid7> files older than startup.orphan_reap_age_secs.

Changed

  • ADR-0044 status superseded by ADR-0052; the no-subprocess invariant is narrowed rather than removed.
  • no_subprocess.rego policy scope narrowed: crates/substrate-subprocess/ is whitelisted for tokio::process::Command; all other crates retain the unconditional prohibition; the forbidden high-level crate list (subprocess, duct, xshell, cmd_lib, shell-words) remains global.
  • tokio workspace dependency gains the process feature when the subprocess Cargo feature is enabled.
  • substrate-mcp-server gains the optional subprocess Cargo feature that activates the substrate-subprocess adapter registration.

Security

  • Subprocess stream transit files created with mode 0600 (owner read/write only) — prevents subprocess output leakage to other users on shared hosts (ADR-0033 revision 2, ADR-0054 revision 2).
  • Layer 5 (subprocess sandbox) added to the security model (ADR-0004 amendment): setsid process group isolation, unconditional stripping of LD_PRELOAD/DYLD_INSERT_LIBRARIES/LD_LIBRARY_PATH/DYLD_LIBRARY_PATH, PR_SET_PDEATHSIG(SIGTERM) on Linux, and macOS watchdog pipe.
  • STRIDE expansion for subprocess BC (ADR-0029 amendment) covering new threat identifiers: S-NEW-3 (binary allowlist bypass), T-NEW-6 (env injection via LD_PRELOAD), R-NEW-1 (deny audit trail for spawned binaries), I-NEW-2 (stdout/stderr capture integrity), D-NEW-3/D-NEW-4/D-NEW-5 (subprocess resource exhaustion, quota, and cascade-kill denial paths), E-NEW-1 (privilege escalation via binary allowlist misconfiguration), E-NEW-2 (container escape via unconstrained subprocess).

0.1.0 — 2026-05-23

First public release. Spec-as-source-of-truth phase complete; across 276 Gherkin scenarios, 72 pass, 54 skip (no step definition yet), and 150 fail — all tracked as follow-up work. The release was iterated through a full parallel-subagent audit (code quality, security, architecture, QA, documentation) before the tag landed at its final commit.

Added

Architecture spec (docs/arch/)

  • 49 MADR 4.0 ADRs covering: bounded contexts, crate stack + async zones, security model, error taxonomy, MCP protocol negotiation, async job control-plane, filesystem index, capability adapter factory, SIMD runtime dispatch, no-subprocess policy, local-deploy + codesign, Mermaid spec diagrams (mandatory), MCP Tasks primitive adoption.
  • 64 Gherkin features across 7 bounded contexts plus cross-cutting concerns (job lifecycle, SIMD fallback, capability negotiation, startup contract, internal-error correlation).
  • 11 CUE schemas with // DDD role: headers covering policy config, error catalog, hints grammar, index config, jobs, MCP tool spec, runtime config, security policy, shared kernel, SIMD capability, tool card.
  • 6 Rego policies including hexagonal_layering, no_subprocess, commit_conventions.
  • Structurizr DSL with C4 context + container views (now reflecting the two platform-shim crates).
  • TLA+ formal model for the job registry (JobRegistry.tla).
  • 41 Mermaid diagrams across ADRs and domain READMEs (ADR-0047).

Workspace (crates/)

  • Cargo workspace layout per ADR-0022 with hexagonal layering enforced via Rego policy: substrate-domain (pure ports + value objects), substrate-policy (allowlist + path jail), substrate-config (figment-based TOML loader), seven adapter crates per bounded context, two platform-shim crates (substrate-signal-sys, substrate-fs-index-macos-sys), and substrate-mcp-server composition root.
  • Rust 2024 edition, MSRV 1.85, toolchain pinned to 1.95 via rust-toolchain.toml.
  • rmcp 1.7 with ["server", "transport-io", "macros"] features (no networked transports).
  • Tokio 1.4x multi-thread runtime; no net feature unless the outbound-net Cargo feature is opted in.

Bounded contexts and tools

  • filesystem-query: fs.read, fs.read_dir, fs.find, fs.stat, fs.hash, with platform-native stat tiers (LinuxStatx, MacosGetattrlist).
  • filesystem-mutation: fs.mkdir, fs.write, fs.copy, fs.rename, fs.remove, fs.set_permissions, fs.symlink, fs.touch, with transactional write pattern (<target>.tmp.<uuid7> + atomic rename).
  • process: proc.list, proc.tree, proc.signal — Linux start-time via /proc/uptime + CLK_TCK; macOS via sysctl. PID allowlist blocks {0, 1, 2} (init / kernel threads) regardless of caller.
  • system-info: sys.info, sys.uptime, sys.df, sys.uname, sys.hostname, sys.load_average.
  • text-processing: text.search, text.count_lines, text.head, text.tail with SIMD-accelerated literal search (Teddy when available).
  • archive: archive.tar.create, archive.tar.extract, archive.zip.create, archive.zip.extract, archive.gzip.compress, archive.gzip.decompress, archive.hash.
  • Asynchronous job control-plane (job.list, job.result, job.cancel, job.status) for Bucket C long-running operations, with idempotency-key dedup and TTL-based result GC.

Security

  • Allowlist (TOML, default-deny) → path jail via strict-path + openat2(RESOLVE_BENEATH | NO_SYMLINKS) on Linux, O_NOFOLLOW_ANY on macOS.
  • Real Linux openat2 capability probe; the post-call canonical path is retrieved via /proc/self/fd/<fd> readlink (not lexically reconstructed) so audit trails carry the kernel-resolved path. SYS_OPENAT2 syscall constants gated per architecture (x86_64 / aarch64 / s390x / riscv).
  • Dry-run mandatory for mutations; elicitation form-mode required for every destructive op: fs.remove, fs.rename (gate added), fs.set_permissions (mask covers world-writable + setuid + setgid), proc.signal SIGKILL/SIGTERM/SIGSTOP, archive create/extract.
  • proc.signal gate order is dry-run → elicitation → PID allowlist → existence → delivery (per ADR-0035).
  • Signal safety: SIGPIPE ignored at startup via dedicated substrate-signal-sys crate; blake3 mmap feature disabled to avoid SIGBUS on concurrent truncation.
  • Cancellation: CancellationToken + tokio::select! biased with work as the first arm.
  • Transactional writes with statvfs preflight disk-space guard.
  • deny.toml configured with vulnerability = "deny", unmaintained = "warn", and yanked-crate denial; .gitignore excludes .env* files.

Errors and audit

  • 13 base error codes plus 6 kernel-induced and 7 startup codes; stable SUBSTRATE_<UPPER_SNAKE> form.
  • Every error envelope carries code, message_en_us, recovery_hint (≤150 chars), and correlation_id (UUIDv7, generated at dispatcher entry and threaded through every error returned by the request).
  • Audit-event taxonomy: structured JSON emitted to stderr only; stdout remains the JSON-RPC channel (ADR-0005).
  • Startup-error correlation IDs use Uuid::new_v7, not a deterministic XOR construction.

Tooling

  • spec validate framework wired (CUE, Conftest/Rego, Vale, Structurizr, MADR lint, Mermaid lint, OpenAPI lint, TLC for the TLA+ model).
  • GitHub Actions CI workflow (.github/workflows/ci.yml) covering cargo build / fmt / clippy / nextest and the spec full lane.
  • mise.toml pinning Rust + the tool versions used by the spec framework.

Testing

  • Cucumber test harness in crates/substrate-mcp-server/tests/cucumber.rs with 311+ step definitions matching 64 Gherkin features.
  • Workspace unit suite: 278 tests passing.
  • Property tests via proptest for system-info, text, archive, filesystem-mutation invariants.

Community and metadata

  • CONTRIBUTING.md, CODE_OF_CONDUCT.md (Contributor Covenant 2.1 by reference), SECURITY.md, CHANGELOG.md, LICENSE (dual-license aggregator), and .github/ templates (issue forms, PR template, Dependabot config). GitHub community-profile health: 100%.
  • Cargo.toml workspace metadata, LICENSE-MIT, LICENSE-APACHE identity normalized to Fabricio Archanjo <farchanjo@gmail.com>.
  • Repository links point at github.com/farchanjo/substrate.

Known limitations

  • 150 of 276 Gherkin scenarios are currently failing — mostly fixture bodies for newly-implemented step defs, OS-specific edges, and async cancellation timing. The skip count fell from 148 to 54 in the audit pass, surfacing real failure modes that were previously hidden.
  • macOS notarization pipeline (ADR-0045) is documented but not yet automated end-to-end.
  • TLC model coverage is partial; only the job-registry invariants are checked.
  • scanner/linux.rs carries a TODO(perf) thread::sleep(100ms) on cold-start delta-CPU sampling. The intended fix (startup LazyLock baseline) is deferred to avoid race-introduction risk.