Add public graph-build phase timings and large generated fixtures

[Iron-Ham]

Severity: Medium
Component: benchmarks / observability / performance regression testing
Affected commands: sem impact, sem graph, sem context, sem verify, MCP graph/impact tools
Reproduced on: repeated performance investigations that required ad hoc timing

Summary

Recent performance work has found several real graph-build bottlenecks, but the project still lacks a public, repeatable way to measure large-repo behavior by phase. That makes performance regressions hard to catch without relying on private repositories or one-off local scripts.

We need two things:

1. phase timings for graph/cache commands,
2. generated public fixtures large enough to exercise TS/JS imports, scope resolution, fallback indexing, cache load, incremental rebuild, and cache save.

Reproduction

Today there is no first-class command like this:

SEM_TIMINGS=1 sem impact f1000 --file f1000.ts --file-exts .ts --json >/dev/null

Expected timing buckets would include at least:

• file discovery
• cache open/load
• full entity extraction
• entity lookup/index construction
• import metadata/table construction
• scope resolution
• fallback reference indexing
• fallback edge resolution
• cache save/incremental save
• CLI output serialization

Without these buckets, each investigation has to add temporary instrumentation or infer bottlenecks from source inspection and wall-clock totals.

Observed

The code has useful tests and some benchmark-like tests, but no public fixture generator or stable phase-timing output for large graph builds. Performance reports therefore tend to mix code reading, local timing, and private-repo observations, which are hard for maintainers and contributors to reproduce.

Expected

A contributor should be able to run a public benchmark and produce a timing report that is safe to paste into an issue or PR.

Suggested fix

• Add a generated fixture tool, for example scripts/gen-large-fixture, with knobs for:
  • file count,
  • entities per file,
  • import fan-out,
  • namespace/default/named import mix,
  • nested class/function depth,
  • generated large entity bodies.
• Add an ignored integration test or benchmark target that builds the fixture and records wall time for cold cache, warm cache, and one-file incremental rebuild.
• Add machine-readable timings behind an opt-in flag or environment variable, for example:
  • sem graph --timings --json
  • SEM_TIMINGS=json sem impact ...
• Keep timing output on stderr unless explicitly requested in JSON, so existing stdout contracts remain stable.
• Document the benchmark command in the performance issue template or contributing docs.

Related issues / PRs

• Slow performance on Go monorepos: scope-resolve runs sequentially and graph cache never persists #105 and resolve_with_scopes Pass 1 still serial on large polyglot repos (follow-up to #105) #111 were hard to compare over time because phase timings were not built into the command.
• Consolidate cache and verify behavior #307, Consolidate TypeScript and JavaScript parser resolution #309, Consolidate Swift extraction #310, and Consolidate identity matching #312 include targeted runtime improvements; this issue is about preventing the same class of regression from returning unnoticed.