diff --git a/README.md b/README.md index 135b740..f431d40 100644 --- a/README.md +++ b/README.md @@ -17,13 +17,16 @@ Latest milestone: [v0.4.0 — second demo and portfolio integration](https://git | [ai-assisted-detection-demo](demos/ai-assisted-detection-demo/README.md) | JSONL auth / web / process | Normalize
Rules
Grouping
ATT&CK mapping | JSON-only case drafting | `rule_hits.json`
`case_bundles.json`
`case_summaries.json`
`case_report.md`
`audit_traces.jsonl` | Human verification required
No autonomous response
No final verdict | ## What This Repo Is - -`telemetry-window-demo` is a local Python CLI that turns timestamped event streams into: - -- sliding-window feature tables -- cooldown-reduced rule-based alerts -- PNG timeline plots -- machine-readable run summaries + +`telemetry-lab` is a small portfolio repository for telemetry analytics and constrained detection-oriented workflows. It is organized as two local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use. + +### telemetry-window-demo + +`telemetry-window-demo` turns timestamped event streams into sliding-window feature tables, cooldown-reduced rule-based alerts, PNG timeline plots, and machine-readable run summaries. + +### ai-assisted-detection-demo + +`ai-assisted-detection-demo` uses deterministic normalization, detection, case grouping, and ATT&CK mapping, then limits the LLM to JSON-only case summarization. Human verification is required, there is no autonomous response, and the demo does not produce a final incident verdict. ## Quick Run diff --git a/docs/ai-assisted-detection-reviewer-pack.md b/docs/ai-assisted-detection-reviewer-pack.md new file mode 100644 index 0000000..2193371 --- /dev/null +++ b/docs/ai-assisted-detection-reviewer-pack.md @@ -0,0 +1,69 @@ +# AI-Assisted Detection Reviewer Pack + +This page describes the small, sanitized reviewer pack for `ai-assisted-detection-demo`. + +Pack directory: + +- `docs/reviewer-pack-v0.4.0/` + +The pack is suitable for attaching to the existing `v0.4.0` release or downloading for offline review. It does not contain secrets, machine-specific paths, or production claims. + +## Pack Contents + +| File | Path | What It Proves | +| --- | --- | --- | +| `MANIFEST.md` | artifact index | The pack scope, source type, and intended reviewer use | +| `accepted-summary-example.json` | accepted path | Accepted output stays constrained and is auditable | +| `rejected-summary-example.json` | rejected path | Rejected output is fail-closed and still recorded | +| `degraded-coverage-example.json` | degraded coverage path | Rule metadata failure is surfaced without changing demo semantics | + +## Accepted Path + +Relevant file: + +- `docs/reviewer-pack-v0.4.0/accepted-summary-example.json` + +Inspect: + +- `case_id` +- `human_verification` +- `scope_guardrail` +- `audit_record.validation_status` + +Reviewer conclusion: + +- Accepted summaries remain bounded to real cases and keep the same operator-review constraints as the demo itself. + +## Rejected Path + +Relevant file: + +- `docs/reviewer-pack-v0.4.0/rejected-summary-example.json` + +Inspect: + +- `summary_status` +- `rejection_reason` +- `audit_record.validation_status` + +Reviewer conclusion: + +- Disallowed or mismatched model output is rejected rather than cleaned up or silently accepted. + +## Degraded Coverage Path + +Relevant file: + +- `docs/reviewer-pack-v0.4.0/degraded-coverage-example.json` + +Inspect: + +- `run_integrity.coverage_degraded` +- `run_integrity.rejected_rules` +- `audit_record.case_id` +- `audit_record.rejection_reason` + +Reviewer conclusion: + +- Rule-metadata failure degrades coverage safely and is disclosed in reviewer-facing artifacts. + diff --git a/docs/release-v0.4.0.md b/docs/release-v0.4.0.md index 6e365d9..844ec43 100644 --- a/docs/release-v0.4.0.md +++ b/docs/release-v0.4.0.md @@ -41,6 +41,7 @@ The repository now presents a two-demo structure: - rejected summary path - degraded coverage path - Added an explicit lifecycle contract and audit schema version in the design documentation +- Added a small sanitized reviewer pack guide in [docs/ai-assisted-detection-reviewer-pack.md](ai-assisted-detection-reviewer-pack.md) ## Why this milestone matters diff --git a/docs/reviewer-pack-v0.4.0/MANIFEST.md b/docs/reviewer-pack-v0.4.0/MANIFEST.md new file mode 100644 index 0000000..6790ac5 --- /dev/null +++ b/docs/reviewer-pack-v0.4.0/MANIFEST.md @@ -0,0 +1,12 @@ +# v0.4.0 Reviewer Pack Manifest + +This pack is a small, sanitized reviewer artifact set for `ai-assisted-detection-demo`. + +It is intended for release attachment or offline review. The files are portable, contain no secrets, and are scoped to the demo's public portfolio story rather than production operations. + +| File | Path Proven | Source Type | What It Proves | +| --- | --- | --- | --- | +| `accepted-summary-example.json` | accepted summary path | sanitized excerpt from committed demo artifacts | An accepted summary is bound to a real case, keeps `human_verification = required`, and is recorded as `accepted` in the audit record | +| `rejected-summary-example.json` | rejected summary path | representative sanitized example aligned with tests | Invalid model output is rejected fail-closed, omitted from accepted summaries, and still recorded in the audit trail | +| `degraded-coverage-example.json` | degraded coverage path | representative sanitized example aligned with tests | Rule metadata failure reduces coverage safely and is surfaced in both run-integrity reporting and audit output | + diff --git a/docs/reviewer-pack-v0.4.0/accepted-summary-example.json b/docs/reviewer-pack-v0.4.0/accepted-summary-example.json new file mode 100644 index 0000000..5916a7c --- /dev/null +++ b/docs/reviewer-pack-v0.4.0/accepted-summary-example.json @@ -0,0 +1,12 @@ +{ + "artifact_kind": "accepted_summary_example", + "source_note": "Sanitized excerpt from committed demo artifacts.", + "case_id": "CASE-001", + "human_verification": "required", + "scope_guardrail": "no_final_incident_decision|no_rule_changes|no_automated_actions", + "audit_record": { + "schema_version": "ai-assisted-detection-audit/v1", + "validation_status": "accepted", + "rejection_reason": null + } +} diff --git a/docs/reviewer-pack-v0.4.0/degraded-coverage-example.json b/docs/reviewer-pack-v0.4.0/degraded-coverage-example.json new file mode 100644 index 0000000..2ad6190 --- /dev/null +++ b/docs/reviewer-pack-v0.4.0/degraded-coverage-example.json @@ -0,0 +1,26 @@ +{ + "artifact_kind": "degraded_coverage_example", + "source_note": "Representative sanitized example aligned with rule-metadata validation tests. This is not a current default-run artifact.", + "run_integrity": { + "accepted_rules": [ + "AUTH-002", + "PROC-001", + "WEB-001" + ], + "rejected_rules": [ + "AUTH-001" + ], + "coverage_degraded": "yes", + "rejection_reasons": [ + "rule_metadata_validation_failed" + ] + }, + "audit_record": { + "case_id": null, + "rule_ids": [ + "AUTH-001" + ], + "validation_status": "rejected", + "rejection_reason": "rule_metadata_validation_failed" + } +} diff --git a/docs/reviewer-pack-v0.4.0/rejected-summary-example.json b/docs/reviewer-pack-v0.4.0/rejected-summary-example.json new file mode 100644 index 0000000..f0d548d --- /dev/null +++ b/docs/reviewer-pack-v0.4.0/rejected-summary-example.json @@ -0,0 +1,12 @@ +{ + "artifact_kind": "rejected_summary_example", + "source_note": "Representative sanitized example aligned with rejection-path tests. This is not a current default-run artifact.", + "case_id": "CASE-EX-REJECTED", + "summary_status": "rejected", + "rejection_reason": "case_id_mismatch", + "audit_record": { + "schema_version": "ai-assisted-detection-audit/v1", + "validation_status": "rejected", + "rejection_reason": "case_id_mismatch" + } +}