Skip to content

Commit d44bb3a

Browse files
stacknilstacknil
committed
Add config-change investigation demo
1 parent 2c3ba52 commit d44bb3a

27 files changed

Lines changed: 1589 additions & 22 deletions

‎README.md‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,25 @@
44

55
Small portfolio prototypes for telemetry analytics, monitoring, and detection-oriented signal processing.
66

7-
Latest milestone: [v0.4.0 — second demo and portfolio integration](https://github.com/stacknil/telemetry-lab/releases/latest). See [release notes](docs/release-v0.4.0.md).
7+
Latest milestone: [v0.6.0 — fourth demo and config-change investigation](https://github.com/stacknil/telemetry-lab/releases/latest).
88

99
## Demos
1010

1111
- [telemetry-window-demo](#telemetry-window-demo)
1212
- [ai-assisted-detection-demo](demos/ai-assisted-detection-demo/README.md)
1313
- [rule-evaluation-and-dedup-demo](demos/rule-evaluation-and-dedup-demo/README.md)
14+
- [config-change-investigation-demo](demos/config-change-investigation-demo/README.md)
1415

1516
| Demo | Input | Deterministic core | LLM role | Main artifacts | Guardrails / non-goals |
1617
| --- | --- | --- | --- | --- | --- |
1718
| [telemetry-window-demo](#telemetry-window-demo) | JSONL / CSV events | Windows<br>Features<br>Alert thresholds | None | `features.csv`<br>`alerts.csv`<br>`summary.json`<br>3 PNG plots | MVP only<br>No realtime<br>No case management |
1819
| [ai-assisted-detection-demo](demos/ai-assisted-detection-demo/README.md) | JSONL auth / web / process | Normalize<br>Rules<br>Grouping<br>ATT&CK mapping | JSON-only case drafting | `rule_hits.json`<br>`case_bundles.json`<br>`case_summaries.json`<br>`case_report.md`<br>`audit_traces.jsonl` | Human verification required<br>No autonomous response<br>No final verdict |
1920
| [rule-evaluation-and-dedup-demo](demos/rule-evaluation-and-dedup-demo/README.md) | JSON raw rule hits | Scope resolution<br>Cooldown grouping<br>Suppression reasoning | None | `rule_hits_before_dedup.json`<br>`rule_hits_after_dedup.json`<br>`dedup_explanations.json`<br>`dedup_report.md` | No realtime<br>No dashboard<br>No AI stage |
21+
| [config-change-investigation-demo](demos/config-change-investigation-demo/README.md) | JSONL config changes<br>Policy denials<br>Follow-on events | Normalize<br>Risky-change rules<br>Bounded correlation | None | `change_events_normalized.json`<br>`investigation_hits.json`<br>`investigation_summary.json`<br>`investigation_report.md` | No realtime<br>No dashboard<br>No AI stage |
2022

2123
## What This Repo Is
2224

23-
`telemetry-lab` is a small portfolio repository for telemetry analytics and constrained detection-oriented workflows. It is organized as three local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use.
25+
`telemetry-lab` is a small portfolio repository for telemetry analytics and constrained detection-oriented workflows. It is organized as four local, file-based demos that are reproducible from committed sample data and intentionally scoped for public review rather than production use.
2426

2527
### telemetry-window-demo
2628

@@ -33,6 +35,10 @@ Latest milestone: [v0.4.0 — second demo and portfolio integration](https://git
3335
### rule-evaluation-and-dedup-demo
3436

3537
`rule-evaluation-and-dedup-demo` starts from raw rule hits and makes cooldown behavior legible. It shows which hits were kept, which were suppressed, how scope was resolved, and why repeated hits collapsed into fewer retained alerts.
38+
39+
### config-change-investigation-demo
40+
41+
`config-change-investigation-demo` follows risky configuration changes into bounded follow-on evidence such as policy denials and service signals. It stays deterministic, file-based, and review-oriented, with no added AI stage.
3642

3743
## Quick Run
3844

@@ -45,6 +51,7 @@ Other demo entrypoints:
4551

4652
- `python -m telemetry_window_demo.cli run-ai-demo`
4753
- `python -m telemetry_window_demo.cli run-rule-dedup-demo`
54+
- `python -m telemetry_window_demo.cli run-config-change-demo`
4855

4956
That command reads `data/raw/sample_events.jsonl` and regenerates:
5057

@@ -109,6 +116,9 @@ Cooldown behavior:
109116
## Repo Guide
110117

111118
- [`demos/rule-evaluation-and-dedup-demo/README.md`](demos/rule-evaluation-and-dedup-demo/README.md) explains the third demo and links its committed before/after dedup artifacts
119+
- [`demos/config-change-investigation-demo/README.md`](demos/config-change-investigation-demo/README.md) explains the config-change investigation demo and its committed artifacts
120+
- [`docs/config-change-investigation-reviewer-pack.md`](docs/config-change-investigation-reviewer-pack.md) walks through the v0.6.0 config-change reviewer pack
121+
- [`docs/telemetry-lab-case-study.md`](docs/telemetry-lab-case-study.md) provides resume-ready, interview-ready, and application-facing project framing
112122
- [`docs/sample-output.md`](docs/sample-output.md) summarizes the committed sample artifacts
113123
- [`docs/roadmap.md`](docs/roadmap.md) sketches the next demo directions
114124
- [`data/processed/summary.json`](data/processed/summary.json) captures the default run in machine-readable form

‎demos/config-change-investigation-demo/README.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Config-Change Investigation Demo
2+
3+
This demo is part of `telemetry-lab` and stays intentionally small, local, and reviewer-friendly.
4+
5+
It focuses on deterministic investigation logic for risky configuration changes and nearby evidence. There is no new AI stage in this demo.
6+
7+
## Purpose
8+
9+
The goal is to make one compact config-change investigation path legible from committed sample data.
10+
11+
The demo starts from configuration changes, policy denials, and follow-on telemetry, then:
12+
13+
- normalizes the inputs into shared internal records
14+
- applies deterministic risky-change rules
15+
- attaches nearby supporting evidence using bounded time and shared-system correlation
16+
- writes machine-readable summaries and a short reviewer-facing report
17+
18+
## Quick Start
19+
20+
From the repository root:
21+
22+
```bash
23+
python -m pip install -e .
24+
python -m telemetry_window_demo.cli run-config-change-demo
25+
```
26+
27+
Generated artifacts are written to `demos/config-change-investigation-demo/artifacts/`.
28+
29+
For offline review without running the demo, see [`docs/config-change-investigation-reviewer-pack.md`](../../docs/config-change-investigation-reviewer-pack.md).
30+
31+
## Demo Inputs
32+
33+
- config changes: `data/raw/config_changes.jsonl`
34+
- policy denials: `data/raw/policy_denials.jsonl`
35+
- follow-on events: `data/raw/follow_on_events.jsonl`
36+
- investigation config: `config/investigation.yaml`
37+
38+
The bundled sample includes:
39+
40+
- one risky MFA-related change with nearby denials and follow-on signals
41+
- one risky public-bind change with nearby denials and service events
42+
- one benign config change that should not trigger an investigation
43+
- one risky break-glass change with no nearby supporting evidence inside the bounded window
44+
45+
## Deterministic Correlation
46+
47+
This demo uses a bounded correlation window after each triggering config change.
48+
49+
Evidence is attached only when:
50+
51+
1. `target_system` matches the triggering change
52+
2. the evidence timestamp falls within the configured correlation window after the change
53+
54+
## Expected Artifacts
55+
56+
- `artifacts/change_events_normalized.json`
57+
- `artifacts/investigation_hits.json`
58+
- `artifacts/investigation_summary.json`
59+
- `artifacts/investigation_report.md`
60+
61+
## Artifact Semantics
62+
63+
- `change_events_normalized.json`: normalized config changes before any rule match is applied
64+
- `investigation_hits.json`: full investigation records, including the triggering change and attached evidence
65+
- `investigation_summary.json`: reduced machine-readable summaries for each investigation
66+
- `investigation_report.md`: a short reviewer report showing the trigger, evidence counts, and bounded-correlation explanation
67+
68+
## Reviewer Walkthrough
69+
70+
1. Open `change_events_normalized.json` and identify the risky config keys and values.
71+
2. Open `investigation_hits.json` and verify which changes became investigations and which evidence records were attached.
72+
3. Open `investigation_summary.json` and confirm the final summaries stay deterministic and bounded.
73+
4. Open `investigation_report.md` and verify that a risky change with no nearby evidence remains explicit rather than silently discarded.
74+
75+
## Limitations
76+
77+
- synthetic sample data only
78+
- no realtime ingestion or service deployment
79+
- bounded correlation by system and time only
80+
- no model-generated reasoning or autonomous response

‎demos/config-change-investigation-demo/artifacts/change_events_normalized.json‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
[
2+
{
3+
"change_id": "cfg-001",
4+
"timestamp": "2026-03-22T09:00:00Z",
5+
"actor": "ops-admin",
6+
"target_system": "identity-proxy",
7+
"config_key": "disable_admin_mfa",
8+
"old_value": "false",
9+
"new_value": "true",
10+
"change_result": "success",
11+
"change_ticket": "CHG-1001"
12+
},
13+
{
14+
"change_id": "cfg-002",
15+
"timestamp": "2026-03-22T09:20:00Z",
16+
"actor": "deploy-bot",
17+
"target_system": "payments-api",
18+
"config_key": "public_bind_cidr",
19+
"old_value": "10.20.0.0/24",
20+
"new_value": "0.0.0.0/0",
21+
"change_result": "success",
22+
"change_ticket": "CHG-1002"
23+
},
24+
{
25+
"change_id": "cfg-003",
26+
"timestamp": "2026-03-22T09:40:00Z",
27+
"actor": "deploy-bot",
28+
"target_system": "payments-api",
29+
"config_key": "log_level",
30+
"old_value": "info",
31+
"new_value": "debug",
32+
"change_result": "success",
33+
"change_ticket": "CHG-1003"
34+
},
35+
{
36+
"change_id": "cfg-004",
37+
"timestamp": "2026-03-22T10:00:00Z",
38+
"actor": "sre-admin",
39+
"target_system": "vault-gateway",
40+
"config_key": "break_glass_mode",
41+
"old_value": "disabled",
42+
"new_value": "enabled",
43+
"change_result": "success",
44+
"change_ticket": "CHG-1004"
45+
}
46+
]

‎demos/config-change-investigation-demo/artifacts/investigation_hits.json‎

Lines changed: 142 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,142 @@
1+
[
2+
{
3+
"investigation_id": "CCI-001",
4+
"severity": "critical",
5+
"rule_id": "cfg_disable_admin_mfa",
6+
"target_system": "identity-proxy",
7+
"actor": "ops-admin",
8+
"triggering_change": {
9+
"change_id": "cfg-001",
10+
"timestamp": "2026-03-22T09:00:00Z",
11+
"actor": "ops-admin",
12+
"target_system": "identity-proxy",
13+
"config_key": "disable_admin_mfa",
14+
"old_value": "false",
15+
"new_value": "true",
16+
"change_result": "success",
17+
"change_ticket": "CHG-1001"
18+
},
19+
"trigger_reason": "Admin MFA was disabled on a protected system.",
20+
"correlation_window_minutes": 15,
21+
"bounded_correlation_reason": "Attached evidence shares target_system 'identity-proxy' and falls within 15 minutes after the triggering change.",
22+
"attached_policy_denials": [
23+
{
24+
"denial_id": "den-001",
25+
"timestamp": "2026-03-22T09:04:00Z",
26+
"actor": "ops-admin",
27+
"target_system": "identity-proxy",
28+
"policy_name": "admin-login-guard",
29+
"decision": "denied",
30+
"reason": "MFA policy blocked admin login after configuration drift."
31+
},
32+
{
33+
"denial_id": "den-002",
34+
"timestamp": "2026-03-22T09:09:00Z",
35+
"actor": "service-account",
36+
"target_system": "identity-proxy",
37+
"policy_name": "token-exchange-guard",
38+
"decision": "denied",
39+
"reason": "Token exchange blocked after admin-auth policy divergence."
40+
}
41+
],
42+
"attached_follow_on_events": [
43+
{
44+
"event_id": "fo-001",
45+
"timestamp": "2026-03-22T09:05:00Z",
46+
"target_system": "identity-proxy",
47+
"event_type": "auth_fail_burst",
48+
"details": "5 privileged login failures from 203.0.113.24 after the config change."
49+
},
50+
{
51+
"event_id": "fo-002",
52+
"timestamp": "2026-03-22T09:11:00Z",
53+
"target_system": "identity-proxy",
54+
"event_type": "service_restart",
55+
"details": "identity-proxy restarted after an auth-policy reload."
56+
}
57+
],
58+
"evidence_counts": {
59+
"policy_denials": 2,
60+
"follow_on_events": 2
61+
}
62+
},
63+
{
64+
"investigation_id": "CCI-002",
65+
"severity": "high",
66+
"rule_id": "cfg_public_bind_cidr",
67+
"target_system": "payments-api",
68+
"actor": "deploy-bot",
69+
"triggering_change": {
70+
"change_id": "cfg-002",
71+
"timestamp": "2026-03-22T09:20:00Z",
72+
"actor": "deploy-bot",
73+
"target_system": "payments-api",
74+
"config_key": "public_bind_cidr",
75+
"old_value": "10.20.0.0/24",
76+
"new_value": "0.0.0.0/0",
77+
"change_result": "success",
78+
"change_ticket": "CHG-1002"
79+
},
80+
"trigger_reason": "Public bind CIDR was expanded to all addresses.",
81+
"correlation_window_minutes": 15,
82+
"bounded_correlation_reason": "Attached evidence shares target_system 'payments-api' and falls within 15 minutes after the triggering change.",
83+
"attached_policy_denials": [
84+
{
85+
"denial_id": "den-003",
86+
"timestamp": "2026-03-22T09:23:00Z",
87+
"actor": "deploy-bot",
88+
"target_system": "payments-api",
89+
"policy_name": "public-exposure-guard",
90+
"decision": "denied",
91+
"reason": "Public bind CIDR exceeded the approved network range."
92+
}
93+
],
94+
"attached_follow_on_events": [
95+
{
96+
"event_id": "fo-003",
97+
"timestamp": "2026-03-22T09:26:00Z",
98+
"target_system": "payments-api",
99+
"event_type": "service_restart",
100+
"details": "payments-api restarted after listener rebind."
101+
},
102+
{
103+
"event_id": "fo-004",
104+
"timestamp": "2026-03-22T09:31:00Z",
105+
"target_system": "payments-api",
106+
"event_type": "edge_warning",
107+
"details": "Edge listener observed requests from the newly public CIDR."
108+
}
109+
],
110+
"evidence_counts": {
111+
"policy_denials": 1,
112+
"follow_on_events": 2
113+
}
114+
},
115+
{
116+
"investigation_id": "CCI-003",
117+
"severity": "high",
118+
"rule_id": "cfg_break_glass_mode",
119+
"target_system": "vault-gateway",
120+
"actor": "sre-admin",
121+
"triggering_change": {
122+
"change_id": "cfg-004",
123+
"timestamp": "2026-03-22T10:00:00Z",
124+
"actor": "sre-admin",
125+
"target_system": "vault-gateway",
126+
"config_key": "break_glass_mode",
127+
"old_value": "disabled",
128+
"new_value": "enabled",
129+
"change_result": "success",
130+
"change_ticket": "CHG-1004"
131+
},
132+
"trigger_reason": "Break-glass mode was enabled on a sensitive service.",
133+
"correlation_window_minutes": 15,
134+
"bounded_correlation_reason": "Attached evidence shares target_system 'vault-gateway' and falls within 15 minutes after the triggering change.",
135+
"attached_policy_denials": [],
136+
"attached_follow_on_events": [],
137+
"evidence_counts": {
138+
"policy_denials": 0,
139+
"follow_on_events": 0
140+
}
141+
}
142+
]

‎demos/config-change-investigation-demo/artifacts/investigation_report.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Config-Change Investigation Demo Report
2+
3+
This deterministic demo correlates risky configuration changes with bounded follow-on evidence.
4+
It does not use an LLM and does not produce autonomous response actions.
5+
6+
## Run Summary
7+
8+
- normalized_change_events: 4
9+
- risky_change_hits: 3
10+
- investigations: 3
11+
- correlation_window_minutes: 15
12+
13+
## CCI-001
14+
15+
- Severity: critical
16+
- Target system: identity-proxy
17+
- Triggering change: cfg-001 (disable_admin_mfa -> true)
18+
- Trigger reason: Admin MFA was disabled on a protected system.
19+
- Attached policy denials: 2
20+
- Attached follow-on events: 2
21+
- Bounded correlation: Attached evidence shares target_system 'identity-proxy' and falls within 15 minutes after the triggering change.
22+
23+
Policy denials:
24+
- den-001: admin-login-guard -> MFA policy blocked admin login after configuration drift.
25+
- den-002: token-exchange-guard -> Token exchange blocked after admin-auth policy divergence.
26+
27+
Follow-on events:
28+
- fo-001: auth_fail_burst -> 5 privileged login failures from 203.0.113.24 after the config change.
29+
- fo-002: service_restart -> identity-proxy restarted after an auth-policy reload.
30+
31+
## CCI-002
32+
33+
- Severity: high
34+
- Target system: payments-api
35+
- Triggering change: cfg-002 (public_bind_cidr -> 0.0.0.0/0)
36+
- Trigger reason: Public bind CIDR was expanded to all addresses.
37+
- Attached policy denials: 1
38+
- Attached follow-on events: 2
39+
- Bounded correlation: Attached evidence shares target_system 'payments-api' and falls within 15 minutes after the triggering change.
40+
41+
Policy denials:
42+
- den-003: public-exposure-guard -> Public bind CIDR exceeded the approved network range.
43+
44+
Follow-on events:
45+
- fo-003: service_restart -> payments-api restarted after listener rebind.
46+
- fo-004: edge_warning -> Edge listener observed requests from the newly public CIDR.
47+
48+
## CCI-003
49+
50+
- Severity: high
51+
- Target system: vault-gateway
52+
- Triggering change: cfg-004 (break_glass_mode -> enabled)
53+
- Trigger reason: Break-glass mode was enabled on a sensitive service.
54+
- Attached policy denials: 0
55+
- Attached follow-on events: 0
56+
- Bounded correlation: Attached evidence shares target_system 'vault-gateway' and falls within 15 minutes after the triggering change.
57+
58+
No nearby supporting evidence fell inside the bounded correlation window.

0 commit comments

Comments
 (0)