feat(runtime): lazy-install matching ABI runtime tree for cross-Python roar run

[christophergeyer]

Summary

Stacked on #107. Closes the loop on the cross-Python roar run story: when the traced Python's ABI doesn't match roar-cli's bundled deps, lazy-install a matching runtime tree at `~/.cache/roar/runtime//` and prepend it to `ROAR_RUNTIME_PYTHONPATH`. The traced process picks up ABI-correct `pydantic_core` / `blake3` from the cache, and the sitecustomize gate (whose detection is also refactored here) sees a matching SO on `sys.path` and lets backend dispatch proceed.

Trigger: `roar run`, not `roar init`. Init-time prefetch was tempting but the user might never invoke Python; `roar run` is the first moment a concrete target exists.

User-visible behavior

• Cache miss — one `🦖 installing roar runtime for cp312 …` line on stderr, then the run proceeds.
• Cache hit — silent. Stamp checks pin the cache to the installed roar version, so a roar upgrade invalidates.
• `ROAR_RUNTIME_INSTALL=skip` (or `roar config set runtime.install skip`) — never lazy-install. Bundled-only. Useful in restricted-network containers; the docs companion PR (`treqs-inc/glaas-site#79`) recommends `pip install roar-cli` as the cleaner pattern for that case.
• Install failure (no network, no `uv`/`pip` available, etc.) — falls through to the gate from fix(inject): gate backend dispatch when traced Python ABI != bundled #107. Stderr shows the gate's actionable message; file I/O still captured.

What changed

• `roar/execution/runtime/abi_probe.py` (new) — subprocess-probes the target Python's `sys.implementation.cache_tag`. Bails for non-Python targets (`bash`, `make`, etc.) so unrelated `roar run` invocations don't pay the probe cost.
• `roar/execution/runtime/lazy_install.py` (new) — XDG-respecting cache, atomic install via `uv pip install --target --python` (plain `pip` fallback), roar-version-stamped invalidation. `ensure_runtime(...)` is the orchestrator that callers (currently just the tracer launcher) use.
• `runtime.install` config key — `auto` (default) | `skip`. Env `ROAR_RUNTIME_INSTALL` wins over config.
• `TracerService._lazy_install_runtime_entries(...)` — calls into the probe + ensure_runtime right before setting `ROAR_RUNTIME_PYTHONPATH`.

Gate refactor (#107 area)

The original ABI-tag check in `sitecustomize.py` parsed roar's bundled `.so` filenames. That's correct for the bundled-only case but blind to a lazy-installed runtime tree on the same `sys.path`. Replaced with `matching_compiled_pydantic_core(sys.path, expected_soabi)` — walks sys.path for a `pydantic_core/*.so` whose filename matches the running interpreter's SOABI. Composes naturally with lazy-install: a matching SO anywhere satisfies the gate.

The old `bundled_abi_tag` + `abi_minor_version` helpers are retained (still tested) for future use and to keep the `support.py` surface stable.

Test plan

• `test_abi_probe.py` — success / non-Python target / subprocess failure / blank stdout / versioned python names.
• `test_lazy_install.py` — cache layout (XDG + home fallback), stamp invalidation, atomic install with mocked subprocess (real tempdir/rename), failure paths, mode resolution (env / config / default / case), and the `ensure_runtime` decision tree (match / skip / cache-hit / cache-miss-install / install-fail).
• `test_inject_support.py` — `matching_compiled_pydantic_core` (bundled / lazy runtime / mismatch / missing pkg / wrong extension / blank entries).
• Full unit suite: 968 passed, 1 skipped (pre-existing).
• `ruff check` + `ruff format --check` clean.

Out of scope

• Hidden `python -m roar.runtime install --python …` entrypoint for power-user pre-warming. Easy follow-up if needed.
• Cache management UX (`list`, `remove`). Same — follow-up.
• Docs page update for `/docs/installation` describing the lazy-install behavior. Will follow on glaas-site.

Merge sequencing

Stacked on `cg/abi-gate-backend-init` (PR #107). When #107 lands on `main`, this branch rebases cleanly. The diff against #107's branch is the meat of this PR; the diff against main shows both PRs' changes together.

🤖 Generated with Claude Code

[christophergeyer]

Both review points addressed in the latest push (74958a1):

1. roar show @N now reflects the traced child's Python, not roar's host

You were right that the Environment block was misrepresenting which Python ran the user's code — exactly the wrong place for that to drift. Root cause: runtime_collector.collect() was calling platform.python_version() in roar's own process, then writing the result onto the job's metadata regardless of what the traced child actually was.

Fix:

• tracker.write_log() now writes python_version + python_implementation into the inject log, captured from inside the traced child via platform.python_version() / platform.python_implementation().
• PythonInjectData model gains both fields (defaults "" for backward-compat with older saved logs).
• runtime_collector.collect() prefers python_data.python_version / python_data.python_implementation and falls back to host platform.* only when the inject log is empty/silent on them.

After this change a roar run invoked under system python3 (3.12) records Python 3.12.x even when roar-cli itself was installed under 3.13.

2. --force → --reinstall in the remediation message

Fixed. The gate's stderr now reads:

- Reinstall roar-cli under matching Python:
    uv tool install --python python3.12 roar-cli --reinstall

Tests added (existing tracker test now asserts the two new payload keys); 968 unit tests passing.

[christophergeyer]

Both lazy-install blockers fixed in c5a1d27 (just pushed).

1. roar.__version__ exists now

roar/__init__.py declares the canonical __version__ via importlib.metadata.version("roar-cli"). The tracer launcher's try/except Exception is narrowed to ImportError and bumped to warning log level — an import failure here is an internal contract violation, not a user-env thing, and shouldn't hide.

Regression caught by a new package-level test that imports from roar import __version__ and asserts it's a non-empty string.

2. Cache now beats system site-packages

The injection function (now correctly named _prepend_roar_runtime_pythonpath) does sys.path[:0] = new_paths instead of sys.path.append. The cache dir lands at sys.path[0], so the ABI-matched typing_extensions / pydantic_core win over the system's stale copies.

Per your reasoning: the prepend "wins" only when the user's env was already mismatched (else the gate would have passed and lazy-install wouldn't fire) — and in that case roar's copy is the right answer. Documented in the docstring.

Subprocess test added that exercises both the prepend codepath (entries land at sys.path[0..N] in declared order) and the early-return path (when roar is already importable, no-op).

Bonus fixes from your report

• probe_python_abi had except (OSError, SubprocessError) — a test-mocked subprocess.Popen could leak a ValueError out. Broadened to except Exception with a comment that the probe's contract is "tag or None."
• 🦖 message gained a tail so a slow network doesn't read as a hang:

  🦖 installing roar runtime for cp312 ... (one-time per Python; cached)
  

971 unit tests, 1 pre-existing skipped. Ruff clean.

Note on your monkey-patched __init__.py on the test machine: once this lands and you uv tool install --reinstall …, the canonical version will be in place.

[christophergeyer]

Reviewer's catch addressed in `b1cd824` (just pushed).

You were right — the writer/reader asymmetry was the textbook missing half. `tracker.write_log` was writing `python_version` / `python_implementation` into the inject JSON, but `DataLoaderService.load_python_data` wasn't extracting those keys when constructing `PythonInjectData`. They defaulted to `""`, so `runtime_collector` hit the `or platform.python_version()` fallback and stamped roar's host Python on the job. Two missing lines.

Tests added at the seam — the third one is the round-trip you specifically asked for:

• `test_python_identity_keys_flow_through` — reader-side: JSON in, model out, keys present.
• `test_python_identity_keys_default_to_empty_for_older_logs` — back-compat: older inject logs without the keys load cleanly.
• `test_writer_reader_roundtrip_carries_python_identity` — real `RuntimeInjectionTracker.write_log` writes, real `DataLoaderService.load_python_data` reads, asserts `python_version` survives. Would have caught both halves; if either side regresses, this test fails.

In-process can only round-trip with the test runner's interpreter, so it asserts the seam (non-empty, dot-separated version) rather than a specific value. The end-to-end "actual mismatched Python" test you described is still the strongest version — that's worth doing as an integration test, but it needs a second CPython on PATH in CI. Worth following up on after the merge.

974 unit tests, 1 skipped, ruff clean.