Add reproducible public documentation benchmarks

[lzehrung]

Summary

• add deterministic public benchmark scenarios comparing baseline direct reads with Codegraph workflows
• add checked result validation, scenario digests, matrix coverage, and documentation synchronization
• publish the current benchmark results with concise interpretation and explicit limitations
• clarify the analysis pipeline and the exact plain .js/.ts scope of --fast-graph
• diagnose the checked benchmark latency with Node CPU profiles, trace events, native-filesystem comparison, and persistent-session measurements

What the checked benchmark shows

• all expected evidence anchors are present in the fixed fixtures
• direct reads are substantially faster in the checked cold, cache-disabled WSL2 run
• Codegraph reduces the declared workflow from three calls to one
• evidence-anchor completeness measures retrieval coverage, not answer quality

Cost diagnosis

• the checkout is on the Windows-mounted WSL2 /mnt/e filesystem
• CLI startup alone is about 3.42 seconds there, versus about 135 ms from an equivalent native WSL /tmp copy
• the TypeScript explore scenario is about 3.81 seconds on /mnt/e, versus about 248 ms on /tmp
• Node trace events show hundreds of synchronous module-resolution filesystem operations; startup accounts for roughly 78-90% of the four one-shot scenario medians
• repeated explore on one persistent session falls to about 23 ms on /mnt/e and 11 ms on /tmp
• the public table therefore measures real cold CLI workflow latency, but it is not evidence that parsing or graph queries intrinsically take several seconds

Verification

• npm run bench:docs:check
• npx vitest run tests/public-docs-benchmarks.test.ts (23 passed)
• npx vitest run tests/workspace-detection.test.ts (4 passed)
• npm run check: lint, formatting, and build passed; the initial test run hit a generated fixture-cache copy race
• clean rerun: npm run test:fast (178 test files passed; 2,177 tests passed; 1 skipped)

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (21 files)

• README.md
• docs/benchmarks/README.md
• docs/benchmarks/results.example.json
• docs/benchmarks/scenarios.json
• docs/cli.md
• docs/how-it-works.md
• package.json
• scripts/benchmarks/benchmark-contract-lib.mjs
• scripts/benchmarks/run-scenario-lib.mjs
• scripts/benchmarks/run-scenario.mjs
• scripts/benchmarks/summarize-results-lib.mjs
• scripts/benchmarks/summarize-results.mjs
• src/cli/help.ts
• tests/public-docs-benchmarks.test.ts
• tests/samples/benchmarks/mixed-docs-source/docs/request-flow.md
• tests/samples/benchmarks/mixed-docs-source/src/client.ts
• tests/samples/benchmarks/mixed-docs-source/src/transport.ts
• tests/samples/benchmarks/sql-application-review/migrations/001_add_account_status.sql
• tests/samples/benchmarks/sql-application-review/schema.sql
• tests/samples/benchmarks/sql-application-review/src/list-active-accounts.ts
• tests/samples/benchmarks/typescript-service/src/handlers/health.ts
• tests/samples/benchmarks/typescript-service/src/routes.ts
• tests/samples/benchmarks/typescript-service/src/server.ts

---

_{Reviewed by kimi-k2.6-20260420 · Input: 136.4K · Output: 25.4K · Cached: 970.8K}

[lzehrung]

Windows performance handoff is now committed at docs/plans/2026-07-10-windows-performance-handoff.md.

It includes the full WSL diagnosis, exact native Windows setup and PowerShell commands, public benchmark and cache protocols, Node CPU profile and trace-event commands, the persistent-session probe, interpretation gates, expected artifacts, and prior verification state. Continue from commit 2d9c29e.