Skip to content

fix: lazy-install JS parser npm deps on first use#37

Draft
joshbouncesecurity wants to merge 3 commits intoknostic:masterfrom
joshbouncesecurity:fix/issue16-21-js-bootstrap
Draft

fix: lazy-install JS parser npm deps on first use#37
joshbouncesecurity wants to merge 3 commits intoknostic:masterfrom
joshbouncesecurity:fix/issue16-21-js-bootstrap

Conversation

@joshbouncesecurity
Copy link
Copy Markdown
Contributor

Summary

openant parse on a JS/TS repo fails out of the box with Cannot find module 'ts-morph' because nothing in the install flow populates parsers/javascript/node_modules/.

This adds a lazy bootstrap in _parse_javascript that runs npm install once if node_modules/ is missing, mirroring the Go CLI's existing venv bootstrap pattern. Lazy (on first JS parse) rather than eager (at CLI startup) so users who only scan Python/Go repos don't need Node/npm installed.

If npm is missing or the install fails, the helper raises with a clear message pointing at the parser directory.

Addresses item 21 from #16 (does not close the issue).

Test plan

  • First JS parse on a clean checkout: bootstrap runs, npm install completes, parse proceeds.
  • Subsequent JS parses skip the bootstrap (node_modules already present).
  • Python/Go scans on a machine without Node installed: no npm invocation, no failure.
  • npm missing on PATH: clear error message names the parser directory.
  • tests/test_js_parser_bootstrap.py passes (5 unit tests covering the four branches plus the integration surface).

openant parse fails with "Cannot find module 'ts-morph'" when the JS parser's
node_modules directory hasn't been populated — there's no bootstrap step that
runs npm install the way the Go CLI's runtime.go does for the Python venv.

_parse_javascript now checks for parsers/javascript/node_modules/ before
invoking Node, and shells out to npm install once if it's missing. Matches
the venv's "first run installs, later runs are fast" UX, but scoped to
actual JS parser use so Python/Go-only users don't need npm.

Fails with a clear error if npm is not on PATH or install fails.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Use node_modules/.package-lock.json as the install completion sentinel so
  a killed prior install (Ctrl+C, OOM) is correctly retried.
- Serialize concurrent bootstraps with a cross-platform file lock so two
  parallel parses don't both run npm install in the same directory.
- Raise a clearer error when package.json is missing instead of letting npm
  silently produce an empty install.
- Include the reproduction command in npm-install failure messages.
- Add tests for partial-install retry, package.json missing, repro-command
  in error, and lock-serialized re-entry.
- msvcrt.locking() locks at the *current* file position, so opening with
  "a+" (which seeks to EOF) made each caller lock a different byte and the
  lock was effectively non-exclusive. Open with "w" and seek to 0 before
  locking so all callers contend on the same byte.
- Add libs/openant-core/parsers/javascript/.openant-npm-install.lock to
  .gitignore so the lockfile isn't accidentally committed.
@joshbouncesecurity
Copy link
Copy Markdown
Contributor Author

Manual verification

  • Fresh checkout (no parsers/javascript/node_modules/): openant parse <ts-repo> prints "installing deps (first run, this may take a minute)..." then proceeds; node_modules/ populated.
  • Second JS parse: skips the bootstrap (no install message).
  • Partial install: rm parsers/javascript/node_modules/.package-lock.json to simulate a Ctrl+C mid-install. Next parse re-runs npm install (sentinel detection).
  • Concurrent first-parse: start two openant parse <js-repo> simultaneously. Lockfile serializes them; only one runs install; second waits then proceeds. No node_modules/ corruption.
  • Missing npm: PATH without npm: clear error message includes "you can reproduce with: npm install (from )".
  • Missing package.json: rare but covered — early error rather than opaque npm failure.
  • Python-only / Go-only scan on a machine without Node installed: no npm invocation, no error.

@joshbouncesecurity
Copy link
Copy Markdown
Contributor Author

Local test results

Tested the lazy bootstrap on Windows by importing core.parser_adapter directly from this branch's worktree (which had no parsers/javascript/node_modules/).

Commands run:

# Branch worktree shipped without node_modules
ls libs/openant-core/parsers/javascript/node_modules
# -> No such file or directory

python -c "
from core import parser_adapter as pa
print('Before:', pa._js_deps_installed())
pa._ensure_js_parser_dependencies()
print('After:', pa._js_deps_installed())
"

First-call output:

[Parser] Installing JS parser dependencies (first run, this may take a minute)...
added 14 packages, and audited 15 packages in 2s
Before: False
Calling _ensure_js_parser_dependencies...
After: True

Second-call output (run immediately after):

Already installed: True
After (no-op): True

Outcome:

  • Fresh checkout (no node_modules/): bootstrap printed Installing JS parser dependencies (first run, this may take a minute)... and ran npm install
  • node_modules/ populated after the call (ts-morph, typescript, etc. present) ✅
  • Second invocation skipped silently — no install message, sentinel returns True ✅
  • Did not exercise missing-npm, partial-install (.package-lock.json removed), concurrent-first-parse, or missing-package.json paths — those are covered by tests/test_js_parser_bootstrap.py in the diff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant