Enforce no-import / no-exec trust invariant across all adapters (M3) (#79)

* Enforce no-import / no-exec trust invariant across all adapters (M3)

Adds two layers of structural enforcement for the public claim in
README and STABILITY.md that the scanner does not execute or import
user code.

1. tests/test_adapter_static_only.py (new) — AST scan of every source
   under src/agents_shipgate/inputs/. Rejects any call to exec / eval /
   __import__ / compile, any use of importlib.import_module /
   importlib.util.spec_from_file_location / importlib.util.module_from_spec /
   importlib.machinery.SourceFileLoader / runpy.run_path / runpy.run_module /
   subprocess.{run,call,Popen,check_call,check_output} / os.{system,popen,
   execv,execvp,spawnv}, and the matching import / from-import forms so an
   aliased re-export can't slip past the call-site checks. Includes negative-
   control parametrize over every forbidden shape and a false-positive
   control proving re.compile / ast.parse / yaml.safe_load / json.loads stay
   green. Baseline scan of current inputs/ is clean — no remediation work
   required to merge.

2. tests/test_fixture_no_import.py (rewritten) — per-adapter live-load
   tests. Each of LangChain, CrewAI, OpenAI Agents SDK, Google ADK, MCP,
   OpenAPI, Anthropic, OpenAI API, and n8n is driven through run_scan()
   against a fixture whose Python content (or sibling trap.py, for
   declarative adapters) raises RuntimeError at module load. Each test
   additionally snapshots sys.modules before/after and asserts no module
   whose __file__ resolves under the fixture root ends up cached — a
   strictly stronger property than relying on the runtime raise alone.
   Sample-protection assertions keep samples/simple_langchain_agent/agent.py
   and samples/simple_crewai_agent/crew.py faithful demonstrators of the
   no-import claim.

3. .github/workflows/ci.yml — adds a dedicated "Trust-model invariant
   lint" CI step that runs the AST scan before the main test suite, so a
   regression is visible at the top of CI logs rather than buried in
   pytest output.

4. STABILITY.md — upgrades the "Trust-model invariants" section to cite
   both tests as the structural enforcement and frame the property as
   "structurally absent" rather than "we tried to forbid X."

42 tests pass; ruff clean. Pre-existing CI surface is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Close lint bypass holes flagged in PR review (M3 follow-up)

PR reviewer found that the v1 lint could be bypassed by:

  1. Module aliasing: `import os as oo; oo.system(...)` — root Name
     was checked verbatim, never resolved to `os`.
  2. From-import alias: `from os import system as sh; sh(...)` — call
     site was a bare Name, not checked against from-import aliases.
  3. `os.exec*` / `os.spawn*` / `os.posix_spawn*` family variants —
     only `execv` / `execvp` / `spawnv` were enumerated; `execve`,
     `execvpe`, `execlpe`, `spawnvp`, `spawnvpe`, `posix_spawn`,
     `posix_spawnp` all slipped through.
  4. Builtins re-export: `import builtins; builtins.eval(...)` — the
     attribute chain wasn't in the forbidden set and `builtins` wasn't
     a forbidden import.

All four bypasses reproduced empty-violations against the v1 lint
before this commit; all are now caught and have dedicated negative-
control parametrize entries.

Implementation changes in tests/test_adapter_static_only.py:

- Two-pass `_scan_source`: pass 1 walks every `Import` / `ImportFrom`
  to build `module_aliases: local -> canonical_module_path` and
  `name_aliases: local -> (canonical_module, canonical_attr)`; pass 2
  resolves attribute chains and bare-name calls through the alias maps
  before checking the forbidden sets.
- New `FORBIDDEN_ATTR_CALL_PREFIXES = ("os.exec", "os.spawn",
  "os.posix_spawn")` catches every variant in those families without
  enumeration. The `os.execv` / `os.execvp` / `os.spawnv` exact entries
  are removed (covered by prefix).
- New `TRACKED_NON_FORBIDDEN_MODULES = {"os"}` records aliases for
  modules that have legitimate uses (`os.path`, `os.environ`,
  `os.getcwd`) but also forbidden surfaces.
- `builtins` added to `FORBIDDEN_MODULES`. `import builtins`,
  `import builtins as b`, `from builtins import eval`, and
  `from builtins import eval as e` are all rejected at the import line.
- From-imports of forbidden attributes are flagged at the import line
  itself, not only at the eventual call site, giving a clearer error.
- Wildcard from-import from a tracked module
  (`from os import *`) is rejected — would otherwise alias forbidden
  names into local scope and defeat the call-site check.
- Acknowledged limitations documented in the module docstring: flow-
  sensitive aliasing (`ev = eval; ev(...)`), reflective access
  (`getattr(os, "system")`, `globals()["eval"]`), and attribute paths
  through aliased submodules. These need code review, not lint.

Negative-control parametrize grows from 11 to 35 cases, covering every
bypass pattern in the docstring. Safe-control test cleans up the prior
`importlib` workaround and now asserts a fully-empty violation list.

65 tests pass (up from 42); ruff clean. Production code under
src/ is unchanged. Baseline scan of current inputs/ remains clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix two more lint holes flagged in PR review (M3 follow-up 2)

Two findings from the second-round review:

  [P2] The two-pass alias map was last-write-wins, so a later
  ``import pathlib as os`` erased the earlier ``import os`` binding
  for the call-site check. ``import os; os.system("ls"); import
  pathlib as os`` therefore passed the lint with zero violations.
  Same shape with ``import os as runner; runner.execve(...); import
  pathlib as runner`` also passed.

  [P3] STABILITY.md claimed ``import importlib*`` forms are
  forbidden, but ``import importlib.metadata`` and ``from
  importlib.metadata import version`` are intentionally allowed
  (the plugin registry under checks/ uses metadata for entry-point
  discovery). The documentation overpromised what the lint enforces.

Reproduced both findings against the prior commit (rebind-os and
rebind-aliased returned []; importlib.metadata returned []) before
fixing.

Implementation (tests/test_adapter_static_only.py):

- Convert ``module_aliases: dict[str, str]`` to
  ``dict[str, set[str]]`` and ``name_aliases:
  dict[str, tuple[str, str]]`` to ``dict[str, set[tuple[str, str]]]``.
  A local name binds to the *union* of every canonical module/attr
  pair it was ever assigned in the file. ``import os; import pathlib
  as os`` now yields ``module_aliases["os"] = {"os", "pathlib"}``.
- ``_resolve_attribute_chain`` -> ``_resolve_attribute_chain_all``
  returns every canonical chain a call could resolve to. Call sites
  flag once if *any* resolution is forbidden.
- Same conservative-union logic for from-import aliases at bare-name
  call sites.
- Added 4 negative-control parametrize cases covering both
  rebind shapes (module alias + from-import alias) plus the
  reverse-order variant where the dangerous binding follows the safe
  one. Verified ``import os; import pathlib as os;
  os.path.join('a','b')`` does NOT false-positive — ``os.path.join``
  is in neither the exact nor prefix sets under either binding.

False-positive trade-off (acknowledged in the docstring): if a local
name is ever bound to a dangerous module *in any branch*, every
attribute chain on that name is checked against the forbidden surface.
For trust-model lint this is the right failure mode — suspicious
aliasing should be a code-review trigger, not a silent pass.

Documentation (STABILITY.md):

- Replace the vague ``import runpy / import subprocess / import
  importlib*`` line with an enumerated forbidden-modules list
  (``runpy``, ``subprocess``, ``importlib``, ``importlib.util``,
  ``importlib.machinery``, ``builtins``) and an enumerated
  forbidden-attribute list (including the ``os.exec*`` / ``os.spawn*``
  / ``os.posix_spawn*`` prefix families and ``os.system`` /
  ``os.popen``).
- Explicitly call out that ``importlib.metadata`` is allowed and why
  (plugin-discovery surface lives under ``checks/`` not ``inputs/``,
  and discovery runs against the installed environment, not user
  workspace files).
- Document the conservative union-of-bindings alias resolution so
  reviewers know an aliased rebinding cannot defeat the lint.

69 tests pass (up from 65); ruff clean. Production code under src/
unchanged. Baseline scan of current inputs/ still clean.

Author: pengfei-threemoonslab

.github/workflows/ci.yml

@@ -36,6 +36,14 @@ jobs:
       - name: Verify generated schemas are up to date
         run: python scripts/generate_schemas.py --check
 
+      - name: Trust-model invariant lint (static, no user code execution)
+        run: python -m pytest tests/test_adapter_static_only.py -q
+        # Fails fast and visibly before the full suite when an adapter
+        # under src/agents_shipgate/inputs/ introduces exec/eval/__import__/
+        # compile or a dynamic-import surface (importlib/runpy/subprocess).
+        # The companion live-load tests in tests/test_fixture_no_import.py
+        # run as part of the main Test step below.
+
       - name: Test
         run: python -m pytest --cov=agents_shipgate --cov-report=term-missing --cov-fail-under=75
 

STABILITY.md

@@ -186,6 +186,49 @@ The scanner does not, under any circumstances:
 - Invoke LLMs
 - Send telemetry
 
+The no-execute / no-import property is enforced by two complementary
+tests on every CI run, not by convention:
+
+- **[`tests/test_adapter_static_only.py`](tests/test_adapter_static_only.py)** —
+  AST scan of every source under `src/agents_shipgate/inputs/`. The scan
+  rejects:
+  - Bare-name calls to `exec` / `eval` / `__import__` / `compile`.
+  - Attribute calls to `importlib.import_module`,
+    `importlib.util.spec_from_file_location`,
+    `importlib.util.module_from_spec`,
+    `importlib.machinery.SourceFileLoader`,
+    `runpy.run_path`, `runpy.run_module`,
+    `subprocess.{run, call, Popen, check_call, check_output}`,
+    `os.system`, `os.popen`, and every variant under the
+    `os.exec*` / `os.spawn*` / `os.posix_spawn*` prefixes.
+  - Module imports of `runpy`, `subprocess`, `importlib`,
+    `importlib.util`, `importlib.machinery`, and `builtins` — in any
+    `import X`, `import X as Y`, `import X.child`, or
+    `from X.child import …` form.
+  - Wildcard `from os import *`.
+
+  `importlib.metadata` is intentionally allowed: the plugin registry
+  under `checks/` (not `inputs/`) uses it for entry-point discovery,
+  and discovery happens against the *installed* environment, not user
+  workspace files. Aliased re-exports (`import os as oo`,
+  `from os import system as sh`, `import os; import pathlib as os`) are
+  resolved through union-of-bindings alias maps so a later import
+  cannot erase an earlier forbidden binding. The lint runs as a
+  dedicated CI step labeled *Trust-model invariant lint* before the
+  main test suite so a regression is visible at the top of CI logs.
+- **[`tests/test_fixture_no_import.py`](tests/test_fixture_no_import.py)** —
+  per-adapter live-load tests. Each adapter (LangChain, CrewAI, OpenAI Agents
+  SDK, Google ADK, MCP, OpenAPI, Anthropic, OpenAI API, n8n, Codex plugin) is
+  driven against a fixture whose Python content (or a sibling `trap.py`, for
+  declarative adapters) raises `RuntimeError` at module load. Each test
+  additionally snapshots `sys.modules` and asserts no module whose `__file__`
+  resolves under the fixture root ends up cached after the scan — a stronger
+  property than relying on the runtime raise alone.
+
+If a contributor introduces a real need for one of the forbidden surfaces,
+update this section in the same PR. The intent is not "we tried to forbid X"
+— it is that X is *structurally absent* from the scanner's parsing path.
+
 Plugins are off by default. `AGENTS_SHIPGATE_ENABLE_PLUGINS=1` enables loading; `--no-plugins` overrides at the CLI level. When loaded, every plugin is enumerated in `report.loaded_plugins`.
 
 ### Manifest Schema