feat(codegraph,skills): code-retrieval engine + agent tools + skill registry & skills_run (D1–D3) [draft]

[sanil-23]

Summary

• codegraph (src/openhuman/codegraph/) — content-addressed code retrieval: per-(repo, ref) manifests over a shared blob cache keyed by git blob SHA + embedding-model signature; a BM25 ∪ structural-aug-dense seed fused via RRF with a coverage flag. Incremental — only changed blobs are (re)embedded; branch switches / renames are near-free.
• Agent tools codegraph_index / codegraph_search registered in all_tools_with_runtime, so coding subagents can seed retrieval before agentic search. Dense vectors reuse the configured (cloud-default) embedder via new embeddings::provider_from_config.
• Size-gated index modes + index-first. IndexMode {Lexical, Dense}: small repos index BM25-only (no embedding calls — recall saturates there anyway), repos above a file-count threshold (OPENHUMAN_CODEGRAPH_DENSE_MIN_FILES, default 400) add the dense arm. codegraph_search indexes the repo first, synchronously, if it hasn't been indexed; search_ref auto-detects which arm exists (dense → BM25 ∪ dense; lexical → BM25-only, no query-embed round-trip).
• Skills registry (skills/registry.rs) — SkillDefinition = #[serde(flatten)] AgentDefinition + declared [[inputs]]; load_skills merges compile-time builtins with runtime <workspace>/skills/<id>/{skill.toml, SKILL.md} (SKILL.md → the inline prompt).
• openhuman.skills_run(skill_id, inputs) — validates required inputs, then builds a real orchestrator Agent (Agent::from_config_for_agent) and runs a full turn focused by the skill's SKILL.md + the inputs, in the background. Every step (tool call + result, sub-agent lifecycle, iteration) streams live to a per-run log at <workspace>/skills/.runs/<skill>_<UTC-ts>_<run>.log (header = inputs + task prompt; footer = status, duration, final output) via an AgentProgress sink. Returns {run_id, status, skill_id, log}. Running a full turn (not a bare run_subagent) establishes its own context — fixing a latent NoParentContext bug where the old handler spawned a subagent with no parent.
• Fixes a pre-existing main test-build break: config/ops_tests.rs built AutonomySettingsPatch without the autonomy-budget fields added by feat: make autonomy action budget configurable #2499/feat: tighten runtime policy + transport guards v2 #2636 (added ..Default::default()).

Draft / WIP. Engine + registry are unit-tested and the whole lib compiles. skills_run is verified live against a standalone openhuman-core: the orchestrator builds, the turn runs, and steps stream to the run log (header → turn started → iteration 1/10 → footer). The smoke's full tool-by-tool trace is gated only by backend sign-in — a standalone core boots signed-out and the chat provider returns SESSION_EXPIRED (embeddings read the JWT per-call, so codegraph works; the chat path needs an active session). Still to come before un-drafting: coverage on the tool wrappers + the skills_run handler, and a skill_list/get/enable introspection RPC. The openhuman.codegraph_* controller RPC is intentionally omitted — subagents reach codegraph through the tools.

Problem

Coding subagents have no cheap way to locate the right files in a repo — cold-start agentic grep is token-heavy — and there is no mechanism to ship and run a predefined, input-parameterised skill (e.g. an autonomous issue-crusher) on demand.

Solution

• A retrieval seed that the A/B work showed beats raw-code embeddings and BM25 alone: lexical (BM25) ∪ structural-augmentation dense, RRF-fused — content-addressed so it stays cheap and incremental. Exposed as tools the agent calls, with a coverage flag so the agent treats partial indexes as hints and falls back to grep.
• Skills are agent definitions + declared inputs; running one validates the inputs, renders them + the SKILL.md guidelines into the task, and drives the orchestrator (full capability — delegate, codegraph, edit/test) focused on the single task. run_subagent gates on spawn depth only, so spawning the orchestrator at depth 1 is allowed.

Validation — SWE-bench_Lite A/B

The retrieval strategy was settled empirically before building the engine, so the Rust code implements a measured choice, not a guess. A file-level recall harness ran three retrievers over the same SWE-bench_Lite instances / corpus / query (the issue text), scored against the files each gold patch edits.

Setup: SWE-bench_Lite (test), n=18 across 6 repos (requests, flask, pytest, pylint, sphinx, xarray; cap 3/repo), embedder bge-small-en-v1.5. Arms: BM25 (lexical), Dense (raw code), Dense (structural-aug) = path-free signatures + imports + called-symbol names + docstrings embedded instead of raw source.

Metric	BM25 (lexical)	Dense (raw code)	Dense (struct-aug)
recall@1	0.222	0.167	0.167
recall@5	0.500	0.444	0.611
recall@10	0.667	0.556	0.778
recall@20	0.722	0.667	0.778
MRR	0.356	0.280	0.361

Findings that drove the design:

1. Raw-code dense loses to BM25 at every k — embedding raw source is worse than plain lexical.
2. Structural-aug dense beats BM25 at recall@5/10/20 (MRR tied) — the struct-doc carries the intent vocabulary raw code lacks.
3. The two are complementary — 6 instances flip at @10 (4 struct-aug-only, 2 BM25-only). BM25 ∪ struct-aug recall@10 = 1.000 on the 16 winnable instances (0.889 / 18; the 2 misses have their gold file excluded from the corpus → unwinnable by any retriever).

⇒ The locked strategy, and exactly what this engine ships: BM25 ∪ struct-aug → RRF fuse → coverage flag → capped agentic. No raw-code vector index (it loses), no LLM gloss.

The harness is a separate Python A/B prototype (bench/codebase-memory-ab/, not in this PR — it validated the strategy); a Rust recall test driving this crate over the cached instances is a follow-up (needs the cloud embedder / a key, so it can't run in the merge gate).

Performance — indexing speed

An #[ignore]d bench_index_speed harness (env-driven, keyless — injects a zero-latency embedder so the measurement isolates engine overhead: git enumeration + structural extraction + tokenization + SQLite) was run over real repos. It surfaced two bottlenecks, both now fixed in this PR:

• Per-blob fsync — put_blob ran in autocommit under synchronous=FULL, so a cold index did one fsync per file. Fixed: new put_blobs batches the insert in a single transaction + PRAGMA synchronous=NORMAL (safe under WAL for a rebuildable cache).
• One embed call per file — index_ref embedded one doc per call = one network round-trip per file against a cloud embedder. Fixed: it now collects uncached blobs and embeds them in batches (≤128/call).

Engine-only cold index, before → after (zero-latency embedder):

Repo	code files	before	after	speedup
flask	79	272 ms	64 ms	4.3×
pytest	184	703 ms	188 ms	3.7×
sphinx	599	2.18 s	413 ms	5.3×
pylint	1,655	5.14 s	387 ms	13.3×
openhuman	2,841	10.2 s	2.86 s	3.6×

Per-file engine cost dropped from ~3.6 ms to ~0.2–1.1 ms; cloud embed round-trips collapse ~100× (e.g. openhuman 2,841 files → 23 embed calls). Warm re-index (content-addressed, all cache hits) of the unchanged 2,841-file tree is ~37 ms (~78k files/s) — the incremental/branch-switch claim, validated.

Live e2e — real cloud embeddings

Two #[ignore]d integration tests exercise the real cloud provider (embedding-v1, 1024-d, the backend's /openai/v1/embeddings via the app-session JWT — no separate key): cloud_embed_probe (one-string liveness) and index_e2e_cloud (index_ref → search_ref over a real repo, asserting full coverage + non-empty hits). Run keyed to a logged-in workspace; they don't run in CI.

A flask run confirms the end-to-end path and gives the real (embedding-included) wall-time the engine-only table can't:

index : files=81 computed=79  in 3572 ms (embedding incl.)   ← one 79-doc cloud batch dominates
search: coverage=Full  in 360 ms
query : "register blueprint route url rule"  → top hit: src/flask/blueprints.py

So cold-index wall-time is embedding-round-trip-bound (≈3.5 s of the 3.6 s is the single cloud batch; engine was ~64 ms), which is exactly why the batching above matters. The e2e also caught a real bug — fixed here: a file with no extractable structure produced an empty structural doc, and the backend 400s an empty embed input; index_ref now falls back to the lexical tokens so an embed input is never empty (guarded by a StrictEmbedder CI regression test).

At scale (the openhuman repo itself, 2,841 files → ~23 cloud batches): cold index ~58.6 s embedding-included vs ~2.9 s engine-only → ~95 % is the embedding API (~2.5 s per 128-doc batch, ~20.6 ms/doc amortized, linear in file count, no rate-limit/batch-size errors). It's a one-time cost — content-addressed, so warm re-index of the unchanged tree is ~37 ms and a branch switch/pull only re-embeds changed blobs. Search returned Partial coverage (12 oversized files skipped) with the top-5 hits all the codegraph source files for a codegraph-themed query — the BM25 ∪ struct-aug → RRF ranking holding up on a real 2.8k-file repo.

Submission Checklist

• Tests added or updated — 13 CI cargo tests: store roundtrip/dedup/gc/persistence + put_blobs batch/dedup, indexer (content-addressed/incremental over a real temp git repo + StrictEmbedder empty-doc regression + lexical-mode never-embeds regression), search (BM25 rank, RRF, partial-coverage), registry (input validation/render, runtime loader); plus 3 #[ignore]d harnesses (bench_index_speed, cloud_embed_probe, index_e2e_cloud).
• Diff coverage ≥ 80% — WIP: engine + registry covered; the codegraph_* tool wrappers and the skills_run handler are not yet unit-covered — to be added before un-drafting.
• Coverage matrix updated — WIP: rows for the new codegraph/skills features to be added with the coverage pass.
• N/A — no matrix feature IDs to list yet (new subsystem).
• No new external network dependencies — dense embeddings reuse the existing (cloud-default) embeddings provider; no new external dep.
• N/A — touches no release-cut manual-smoke surface (additive tools + an opt-in RPC).
• N/A — no upstream issue to close; tracked by the scope-of-work at sanil-23/openhuman#12.

Impact

• Desktop core only (Rust lib). Additive: new domain codegraph, two agent tools, a skills registry + one RPC. No migrations. Dense retrieval uses the existing cloud embedder (per-repo first-index cost, amortised, content-addressed). codegraph DB lives at <workspace>/codegraph/index.db.

Related

• Closes: (none — scope-of-work tracked at sanil-23/openhuman#12)
• Follow-up PR(s)/TODOs: skill_list/skill_get/skill_enable introspection RPCs; tool/handler coverage; end-to-end skills_run; ship the issue-crusher as a bundled skill.

---

AI Authored PR Metadata

Linear Issue

• Key: N/A
• URL: N/A

Commit & Branch

• Branch: feat/codegraph-skills
• Commit SHA: 768d1b0c

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: be072717-220d-4fb4-8e33-2dcff248d11d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[graycyrus]

@sanil-23 CI is failing (PR Submission Checklist) and there are pending E2E checks, so holding off on a full approve for now. I did skim through though — the engine design is solid and the empirical validation behind the retrieval strategy is a nice touch. A few things while I'm here:

l2_normalize is duplicated — it's defined identically in both codegraph/index.rs and codegraph/search.rs. Pull it into store.rs or a small math.rs in the module and import it from both. Small thing but it'll bite you when you go to tune the normalization behavior.

Developer home path hardcoded in index_e2e_cloud — src/openhuman/codegraph/index.rs has /home/sanil/vezures/openhuman-cbmem-ab/... as the default fallback in the #[ignore]d e2e test. The env-var override works, but the fallback path will be confusing for anyone else running it. Either drop the fallback or use a more generic placeholder.

No dedup on skill IDs in load_skills — builtins are loaded first, then runtime skills are appended. If a runtime skill has the same id as a builtin, get_skill returns the builtin (first match). Whether runtime skills shadow builtins or vice versa should be deliberate — add a comment or a dedup pass so the precedence is explicit.

No status/cancel endpoint for background runs — skills_run fires and forgets; the only feedback is the log file path. You mentioned skill_list/skill_get are follow-up work, so just flagging it as something to track before un-drafting. Clients can't poll or cancel a running skill right now.

Fix the CI, finish the tool wrapper + handler coverage you called out in the checklist, and this is in good shape. Let me know if you hit anything odd.

[graycyrus]

[minor] l2_normalize is identical to the one in index.rs (line 214). Extract it to a shared location in the module.

[graycyrus]

[minor] Default fallback path /home/sanil/vezures/... is a developer-local path. Anyone else running this test with --ignored gets a confusing missing-repo error before the env-var message. Use "." or just remove the fallback entirely and always require CODEGRAPH_E2E_REPO.

[graycyrus]

[minor] Builtins and runtime skills are appended without dedup on id. If a runtime skill.toml declares the same id as a builtin, you get two entries — get_skill returns the builtin (first match) silently. Either document that builtins take precedence, or deduplicate explicitly.