Skip to content

test(security): scope-leak fuzzer — assert no read method leaks artifacts across visibility scopes #328

Description

@plind-junior

scope filtering is arriving piecemeal. src/vouch/scoping.py already provides is_visible(scope, viewer), filter_hits, and filter_audit_events, and kb.search / kb.context / kb.audit thread a ViewerContext through it. but the coverage is uneven: kb.synthesize and kb.neighbors take no project/agent params, and every kb.list_* / kb.read_* handler returns model_dump() straight off KBStore with no scope check at all. a private or wrong-project claim that kb.search correctly hides is still fully readable through kb.list_claims or kb.read_claim. once richer (visibility, project, agent) scopes land end-to-end (#189 / #100), the boundary needs a test that proves it holds across every read surface, not only the three that were wired first.

this issue is that test. it does not implement scoping — it fuzzes it. the target is a single property/fuzz suite that fails loudly the moment a new read method forgets to apply is_visible.

proposed surface

no new kb.* method — this is test-only, under tests/test_scope_leak_fuzz.py (mirrors the module convention).

  • a property-based generator seeds a kb with claims and sources spread across all four Visibility values (public, team, project, private) and a spread of project / agent scope tuples, plus a matching set of ViewerContext callers.
  • a driver enumerates every read method and calls it once per caller scope: kb.search, kb.context, kb.synthesize, kb.neighbors, kb.list_pages, kb.list_claims, kb.list_entities, kb.list_relations, kb.list_sources, kb.read_page, kb.read_claim, kb.read_entity, kb.read_relation, and kb.audit.
  • for each (method, viewer) pair, assert every artifact id in the result satisfies is_visible(artifact.scope, viewer) — zero ids that fail the predicate. audit results additionally assert against event_visible_to_viewer.
  • a registry guard: enumerate the read methods off capabilities.METHODS and fail if any read-side method is absent from the driver's coverage set, so a newly added reader cannot silently escape the sweep.
  • runs in the default pytest tests/ -q job with no embeddings dependency; the search backend under test is substring/fts5, so it stays in the non-embeddings ci gate.

review gate & scope

pure test code — it exercises read paths only, never proposes, approves, or writes approved artifacts, so the review gate is untouched. seeding goes through the normal propose_* → kb.approve path (or a fixture that drives proposals.approve) so the fixtures themselves respect the gate rather than writing yaml directly. everything runs against a temp .vouch/ on local disk with no network, keeping it local-first. the fuzzer asserts on the retrieval/read boundary only and never reaches into storage.py internals.

one known fail-open case the fuzzer must encode as expected behavior, not a bug: is_visible treats a project-visibility artifact whose scope.project is None as visible to everyone, and fails closed for private. the oracle is is_visible itself, so the test tracks the shipped semantics rather than a hand-rolled duplicate.

acceptance criteria

  • tests/test_scope_leak_fuzz.py seeds claims + sources across all four Visibility values and multiple project/agent tuples via the review-gated propose→approve path
  • driver exercises every read method (search, context, synthesize, neighbors, list_*, read_*, audit) under each caller ViewerContext
  • for every (method, viewer) pair, asserts zero result ids fail scoping.is_visible (audit uses event_visible_to_viewer)
  • registry guard fails if a read-side method in capabilities.METHODS is missing from the driver's coverage set
  • currently-unscoped readers (kb.read_*, kb.list_*, kb.synthesize, kb.neighbors) are covered; the test stays red until they filter, green once they do
  • runs in the default non-embeddings pytest tests/ -q gate; mypy src and ruff check stay clean

distinction from adjacent issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    size: M200-499 changed non-doc linesteststests and fixtures

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions