1.0.0Release

RAGLeakLab v1.0.0

RAGLeakLab is now a v1, deterministic security testing framework for RAG systems: measure leakage, detect corpus poisoning, and gate CI pipelines with reproducible packs, strict metrics, and regression baselines.

🚀 Highlights

• Deterministic, CI-first security testing for RAG pipelines (reproducible packs + regression gates).
• Leakage + integrity coverage: semantic & cross-document leakage + safe corpus poisoning detection.
• Production-grade workflows: delta ingestion gates, SARIF/JUnit outputs, triage + annotations.

✨ New in v1

• Threat Packs

  • canary-basic — canary token extraction
  • verbatim-basic — verbatim document leakage measurement
  • membership-basic — membership inference detection
  • semantic-basic — claim-based semantic leakage detection (financial/medical/legal)
  • crossdoc-basic — cross-document synthesis leakage detection
  • sentinel-takeover-safe — corpus poisoning / sentinel takeover detection (safe triggers)

• CI & Automation

  • diff regression gate (non-zero exit on metric regressions)
  • delta ingestion gate for corpus changes (C → C′)
  • Determinism verification: verify determinism
  • Cassette record/replay for network-free, stable CI on HTTP targets

• Analysis & Reporting

  • report summarize (top findings + attribution + remediation hints)
  • report annotate GitHub-style ::error:: / ::warning:: PR annotations
  • Multi-format outputs: JSON, SARIF, JUnit, Markdown
  • Secret redaction (emails, API keys, canaries) in outputs by default
  • Query minimization: --minimize-on-fail

• Developer Experience

  • --jobs N parallel execution with deterministic ordering
  • Plugin system (entry-point plugins for custom metrics/attacks/targets)
  • HTTP target adapter with SSRF protection + domain allowlisting
  • assets validate + config validate (JSON Schema export)

🧱 Breaking changes

• None intended for v1 users. Public contracts and schemas are documented in:
  • docs/V1_CONTRACTS.md
  • docs/REPORT_SCHEMA.md
  • docs/STABILITY.md

📦 Upgrade

uv sync --all-extras

✅ Recommended verification

./scripts/ci_smoke.sh
uv run ragleaklab verify determinism --pack canary-basic --runs 2 --out out/determinism

🔎 Quickstart

uv run ragleaklab run --pack canary-basic --out out/canary
uv run ragleaklab diff --baseline baselines/v1/report.json --current out/report.json
uv run ragleaklab report summarize --in out/ --top 20