Repository navigation
Code Analysis Reliable Investigation
This experimental work makes source investigations more traceable: locate definitions and imports, inventory Python calls with explicit unknowns, and compare selected definitions across two revisions. A fixed evaluation of seven real projects found useful improvements, with substantial limits on call resolution and no claim of runtime behavior or general accuracy.
The work is an unmerged native stack, #1237, consisting of Draft PRs #1235, #1236, and #1243. No PR has been merged; the original 14 contributor branches remain. This is experimental, not a release or live installation.
| Revision | Role |
|---|---|
bf0a29e |
Labs baseline |
7fa8827 |
Combined investigation baseline |
8739a97 |
Python lexical calls |
92606d7 |
Final paired-definition candidate |
Public replay: a fresh GitHub fetch of the pinned analyzer and all 12 target roots passed the source/committed-ZIP replay. The executable bundle and structured receipts are in assets/reliable-investigation/ in this Wiki Git repository. Local tests, public replay, CI and human review remain separate evidence.
CI, 6 November 2026 17:33 UTC: #1235 and #1236 completed with the previously reproduced Labs README motion-proof digest failure and its dependent Node 22 gate; all other executed jobs passed, including Code Analysis and ZIP freshness. #1243 also completed with the same verified failure. Each run has 17 successful jobs, the README regression and dependent Node 22 gate failed, and two deployment jobs skipped. Code Analysis, ZIP freshness, browser/WebM, Windows portability and package smoke passed. CI is not green. The mismatch is expected ca273484a235600f96a74351ac3976cdb6bffb1cfb5d0b86f9ced8c21bd1aa80, actual 7f68d7303c86988cfbe228202d25fc6da42fb174be47df5f637d57c39fbf5766.
The original questions were informed by source-first investigations. The atomic scoring units were frozen after prior analyzer exposure: this is a targeted regression evaluation, not a blind or held-out benchmark. The expected source facts were derived independently of analyzer output. Grading checks selected identities, spans, source hashes and navigation where applicable; manual source reading and runtime observations are not credited as analyzer answers.
| Selected units | Labs | Combined | Python | Final |
|---|---|---|---|---|
| Function locations (8) | 7 correct, 1 omitted | 8 correct | 8 correct | 8 correct |
| Import sites (7) | 7 correct | 7 correct | 7 correct | 7 correct |
| Python callsites (52) | 52 omitted | 52 omitted | 4 targets, 48 unknown | 4 targets, 48 unknown |
| Commander dynamic/member calls (2) | 2 omitted | 2 unknown | 2 unknown | 2 unknown |
| Changed definitions (8) | 8 omitted | 3 correct, 5 omitted | 3 correct, 5 omitted | 8 correct |
| Outgoing import sites (4) | — | — | — | 4 correct |
The final evaluation also records two Java call boundaries as explicitly unavailable, with null totals. The four Python targets match the supported frozen subset; 48 unknowns are not recovered targets. There were no wrong or omitted expected units in the final selected set. The scorer does not inventory every extra assertion across each project, so this is not an all-output precision or zero-false-positive claim.
Eight changed definitions come from four projects: Commander (3), HTTPX (2), pluggy (1), and SLF4J (2). Redux is the fifth paired change, but its selected change is an export, not a scored changed definition. These source facts do not answer runtime questions such as which plugin or provider executes, callback identity, or transitive impact.
Five rows have base and head revisions; Guava and pandas are single-revision source questions. Links point to the exact public commits.
| Project | Base | Head | Selected investigation |
|---|---|---|---|
| Redux | 85d7f27 | c9e0650 | Public export and existing callers |
| Commander | 3ae30a2 | 2c7f687 | Hook registration and dispatch |
| pluggy | 2e916c2 | 3d5417a | Hook filtering and invocation |
| HTTPX | eeb5e3c | e9cabc8 | Deferred imports and constructor paths |
| SLF4J | aa13da3 | 9783320 | Explicit provider selection |
| Guava | — | f06690f | Java Unicode source locations; guava/src scope |
| pandas | — | 0691c5c | Indexing definitions and calls |
The cases cover JavaScript, TypeScript, Python and Java; Ruby and other languages are not covered. Captured and role-filtered file counts do not include files excluded before capture and do not establish whole-repository coverage.
At 92606d7, source and the committed ZIP produced byte-identical raw facts for 12/12 checks, equivalent normalized results for 10/10 saved questions and 5/5 paired questions, and matching exits for all 27 commands. Normalization only removes run-specific raw-facts and workflow paths. This equality checks packaging consistency, not correctness by itself.
Four paired workflows were ready with exit 0. SLF4J returned exit 1 and an incomplete, partial result on both modes: slf4j-migrator/jcl/jclcontent.java is a conversion example containing top-level assignments. The selected definitions remained available and gradeable; the repository scan was not complete.
The final module suite passed 331/331 tests. An independent agent review reran 9 library probe groups and 4 CLI cases; this is not human review. The committed archify.zip has SHA-256 5580acd25f745f34eb4d4f1e8e5f3d2cb0468ff76ba5baeefeb63417378e4ea3; all 153 tracked Code Analysis module files matched the archive. No target application was executed.
The Python design inventories source calls separately from resolving bindings. It resolves only a conservative supported subset of direct names and preserves unresolved cases explicitly. The Python 3.13 execution model explains why function-wide bindings and scope matter; the pinned Pyright binder shows separate handling of function headers; and pinned scip-python import handling illustrates preserving import provenance. These sources informed design choices, not a measured comparison against those tools.
A lexical edge is not a guarantee of runtime dispatch. Decorated targets, dynamic calls, member resolution, reexports and other unsupported binding cases require caution. Java call tracing is unavailable in this pinned 92606d7 composition; changed-definition comparison does not infer renames, moves or runtime impact. The independently developed Java call Draft #1239 has its own evaluation and limitations. It is not included in this matrix; combining the two Java adapter changes still requires revision reconciliation and validation.
The Wiki replay bundle contains the frozen public inputs, scorer and provenance. It fetches the exact source projects and product commit, executes the analyzer from both source and its committed ZIP, and compares receipts; it does not run the target applications. GitHub and npm network access, Git, Node 22.23.2, npm and Python 3.13 are required; install these runtimes separately. Each full run needs a new work directory and space for the checkouts and outputs.
git clone https://github.com/tt-a1i/archify.wiki.git
cd archify.wiki/assets/reliable-investigation
python3.13 replay.py --revision paired --work /path/to/new-replayInspect the preflight, command records, unit grades, paired contract and source/ZIP comparison in the work directory. A partial directory is not a successful run. The bundle's README.md describes the contract and oracle provenance. The recorded run completed on macOS with Node 22.23.2 / Python 3.13.16. See receipts/public-replay-92606.json, receipts/unit-results.json, receipts/baseline-unit-results.json and receipts/source-vs-archive.json in the bundle. Personal execution paths are replaced by placeholders; source facts and outcome statuses are unchanged.
Draft #1252 builds on fixed #1250 without rewriting the retained team contributions. A source-first Click/Jinja sample recovered three missed function-local explicit-import targets: selected correct targets increased from 2 to 5, with 0 wrong targets, 0 omitted calls and 5 expected unknowns in both candidate source and extracted ZIP. The separate validation record contains fixed commits, per-unit evidence, repeatable commands, the rejected initial package, and the retained baseline TypeScript test failure. This follow-up does not replace the historical matrix above or establish general call accuracy.