Skip to content

Commit 3850765

Browse files
System Administratorclaude
andcommitted
Add synchronous fast-path blocklist enforcer (EPP) and update README
Implement a synchronous blocklist evaluator in the processor hot loop that blocks known-bad IPs, CIDRs, domains, process names, file paths, and chain patterns in sub-millisecond time — skipping both graph writes and LLM analysis. README updates: new EPP section, Usage Guide (response modes, recommended workflow, API examples), updated architecture diagram, pipeline flow, security framework (9 defense-in-depth layers), Prometheus metrics, project structure, test count (~550), and corrected default mode (passive). Retake three screenshots with live agent data showing fast-path blocking in action. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 99d095a commit 3850765

11 files changed

Lines changed: 913 additions & 19 deletions

File tree

README.md

Lines changed: 105 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -19,11 +19,11 @@ Built from scratch as a single-developer project. Combines real-time telemetry c
1919
│ │
2020
│ ┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
2121
│ │ Collectors │──▶│ Normalizer │──▶│ Processor │──▶│ Graph DB │ │
22-
│ │ (per-OS) │ │ (OCSF) │ │ (entities) │ │ (Kuzu) │ │
23-
│ └─────────────┘ └──────────────┘ └──────────────┘ └──────┬───────┘ │
24-
│ │ │ │
25-
│ ▼ ▼ │
26-
│ ┌─────────────┐ ┌─────────────────────────────────────────────────────┐ │
22+
│ │ (per-OS) │ │ (OCSF) │ │ (entities + │ │ (Kuzu) │ │
23+
│ └─────────────┘ └──────────────┘ │ fast-path) │ └──────┬───────┘ │
24+
│ │ └──────┬───────┘ │ │
25+
│ ▼ │ (blocked) ▼ │
26+
│ ┌─────────────┐ ┌─────────────────────────────────────────────────────┐ │
2727
│ │ SQLite │ │ LLM Analyzer │ │
2828
│ │ Queue │ │ ┌──────────┐ ┌───────────┐ ┌──────────────────┐ │ │
2929
│ │ + Findings │ │ │ Preflight│─▶│ Tool-Use │─▶│ Finding Builder │ │ │
@@ -39,7 +39,7 @@ Built from scratch as a single-developer project. Combines real-time telemetry c
3939
│ │ │
4040
│ ▼ │
4141
│ ┌───────────────────────────────────────────────────────────────────────┐ │
42-
│ │ Response Engine │ │
42+
│ │ Response Engine ◀── fast-path (skip LLM) │ │
4343
│ │ Severity ──▶ Baseline/Allow/Block ──▶ Approval ──▶ Execute ──▶ Audit │ │
4444
│ │ │ │
4545
│ │ Actions: Suspend │ Terminate │ Isolate Network │ Block IP │ │
@@ -57,7 +57,7 @@ Built from scratch as a single-developer project. Combines real-time telemetry c
5757

5858
1. **Collect** — Platform-native collectors gather process, network, file, DNS, and registry events
5959
2. **Normalize** — Raw events are standardized to OCSF (Open Cybersecurity Schema Framework)
60-
3. **Extract & Graph** — Entities (processes, IPs, domains, files) and relationships are written to a Kuzu property graph
60+
3. **Extract & Enforce** — Entities are extracted and checked against the synchronous fast-path blocklist. Matches are blocked instantly. Non-blocked entities are written to the Kuzu property graph
6161
4. **Analyze** — An LLM with tool-use capabilities investigates novel behaviors using graph context, threat intel, and external APIs
6262
5. **Respond** — A policy engine maps severity to actions, checks baselines/allowlists, requests approval, executes, and audits everything
6363

@@ -163,6 +163,21 @@ A three-mode response engine that maps LLM severity verdicts to automated or sup
163163

164164
**Protected process list** prevents the agent from terminating system-critical processes (`launchd`, `csrss.exe`, `systemd`, `sshd`, etc.) regardless of severity.
165165

166+
### Synchronous Fast-Path Blocklist (EPP)
167+
168+
The fast-path enforcer evaluates blocklist rules **synchronously in the processor pipeline**, immediately after entity extraction — before the event reaches the graph database or LLM analyzer.
169+
170+
**How it works:**
171+
172+
- Blocklist rules are compiled into O(1) in-memory structures: IP hash sets, domain hash sets, CIDR prefix lists, and glob pattern lists
173+
- On every event, entities are checked in evaluation order: **IPs → CIDRs → domains → process names → file paths → chain patterns**
174+
- On match: a CRITICAL finding is generated and the response engine is triggered immediately, **skipping both the graph write and LLM analysis**
175+
- The compiled rule set is thread-safe with periodic SQLite refresh (5s default) and instant invalidation when rules are added or removed via the dashboard
176+
177+
**Why it matters:** Known-bad indicators (C2 IPs, malicious domains, prohibited process chains) are blocked in sub-millisecond time — no waiting for the LLM analysis cycle. This turns the EDR from a detect-and-alert system into a real-time enforcement point for known threats.
178+
179+
The dashboard shows a **fast-blocked event counter** on the Overview tab when events have been blocked by this path.
180+
166181
### User Identity Enrichment
167182

168183
Every process in the graph is linked to the user who spawned it. The agent resolves the owning user for each process via OS-level APIs (`stat /proc/<pid>` on Linux, `ps -o user=` on macOS, token query on Windows) and writes `(:User)-[:SPAWNED]->(:Process)` edges into the graph.
@@ -283,6 +298,7 @@ edr_llm_call_latency_seconds
283298
edr_llm_verdicts_total{severity}
284299
edr_dga_detections_total
285300
edr_persistence_detections_total{type}
301+
edr_events_fast_blocked_total
286302
edr_response_actions_total{action, result}
287303
edr_tamper_detections_total{event_type}
288304
edr_agent_uptime_seconds
@@ -311,19 +327,26 @@ macOS system tray icon provides live status, native notifications for HIGH/CRITI
311327
│ IOC matching │ │ Severity verdict + findings │ │ Approval gate │
312328
│ Code signing │ │ │ │ Action executor │
313329
└──────────────┘ └──────────────────────────────────┘ │ Audit trail │
314-
└──────────────────┘
330+
└────────▲─────────┘
331+
332+
┌───────────────────────────────────────────────────────────────┐ │
333+
│ Fast-Path Blocklist Enforcer (in Processor) │────┘
334+
│ IPs → CIDRs → domains → process names → file paths → chains │
335+
│ O(1) compiled in-memory structures, sub-ms evaluation │
336+
└───────────────────────────────────────────────────────────────┘
315337
```
316338

317339
**Defense-in-depth layers:**
318340

319341
1. **Collection** — Native OS APIs for high-fidelity telemetry (ETW, auditd, FSEvents, Unified Log)
320342
2. **Normalization** — OCSF standardization ensures consistent analysis regardless of platform
321343
3. **Graph Correlation** — Entity relationships reveal multi-step attack patterns invisible in flat logs
322-
4. **Real-Time Detection** — DGA, persistence, and IOC detectors catch known patterns immediately
323-
5. **AI Reasoning** — LLM analyzes novel behaviors with graph context and external intelligence
324-
6. **Response Orchestration** — Graduated actions (log → alert → suspend → terminate → isolate) with approval gates
325-
7. **Behavioral Baseline** — Learning mode builds a profile of normal behavior; active mode only responds to deviations
326-
8. **Self-Protection** — Tamper detection, protected process list, heartbeat monitoring
344+
4. **Real-Time Detection** — DGA, persistence, IOC, and fast-path blocklist detectors catch known patterns immediately
345+
5. **Fast-Path Enforcement** — Synchronous blocklist evaluates IPs, domains, CIDRs, process names, file paths, and chain patterns in the processor hot loop — blocking known threats instantly without LLM analysis
346+
6. **AI Reasoning** — LLM analyzes novel behaviors with graph context and external intelligence
347+
7. **Response Orchestration** — Graduated actions (log → alert → suspend → terminate → isolate) with approval gates
348+
8. **Behavioral Baseline** — Learning mode builds a profile of normal behavior; active mode only responds to deviations
349+
9. **Self-Protection** — Tamper detection, protected process list, heartbeat monitoring
327350

328351
---
329352

@@ -341,7 +364,7 @@ macOS system tray icon provides live status, native notifications for HIGH/CRITI
341364
| Process Info | psutil |
342365
| macOS Tray | rumps |
343366
| Logging | structlog (JSON/text) |
344-
| Testing | pytest (~500 tests) |
367+
| Testing | pytest (~550 tests) |
345368

346369
---
347370

@@ -360,7 +383,72 @@ export DEEPINFRA_API_KEY="your-key-here"
360383
sudo .venv/bin/python3 -m agent.main --config config.yaml --log-level INFO
361384
```
362385

363-
Dashboard opens at `http://localhost:9200`. The agent starts in **learning mode** by default — switch to **active** mode via the Settings tab when ready to enforce response actions.
386+
Dashboard opens at `http://localhost:9200`. The agent starts in **passive** mode by default — switch to **learning** to build a behavioral baseline, then **active** to enforce.
387+
388+
---
389+
390+
## Usage Guide
391+
392+
### Response Modes
393+
394+
The agent operates in one of three modes, switchable at runtime via the dashboard **Settings** tab or the API:
395+
396+
| Mode | What happens on a threat | When to use |
397+
|------|-------------------------|-------------|
398+
| **Learning** | Records all observed behaviors to a baseline. No alerts, no enforcement. | First deployment — build a profile of normal activity before enabling detection. |
399+
| **Passive** | Generates findings and alerts (dashboard + tray notifications). No enforcement actions. | Day-to-day monitoring when you want visibility without automated response. |
400+
| **Active** | Evaluates findings against blocklist → allowlist → baseline → policy, then executes response actions (suspend, terminate, isolate, etc.) with approval gates. | Production enforcement — the agent actively responds to threats. |
401+
402+
### Recommended Workflow
403+
404+
1. **Start in Learning mode** — Let the agent observe normal behavior and build a baseline.
405+
- **Development machines**: 24 hours is usually sufficient
406+
- **Servers / production hosts**: 1–7 days to capture periodic jobs, maintenance windows, and varied workloads
407+
2. **Switch to Passive mode** — Review findings in the dashboard. Add allowlist rules for known-good behaviors that generate false positives. Add blocklist rules for known-bad indicators you want blocked immediately.
408+
3. **Switch to Active mode** — The agent now enforces. Baselined behaviors are silently passed, allowlisted behaviors are skipped, blocklisted behaviors are blocked instantly (via the fast-path enforcer), and novel threats go through the LLM → policy → approval → action pipeline.
409+
410+
### Switching Modes
411+
412+
**Dashboard:** Settings tab → Response Mode dropdown → select mode.
413+
414+
**API:**
415+
```bash
416+
curl -X POST http://localhost:9200/api/response/mode \
417+
-H 'Content-Type: application/json' \
418+
-d '{"mode": "active"}'
419+
```
420+
421+
### Adding Blocklist / Allowlist Rules
422+
423+
**Dashboard:** Settings tab → scroll to Blocklist or Allowlist section → fill in rule type, pattern, and optional chain filter → Add.
424+
425+
**API:**
426+
```bash
427+
# Block a specific IP
428+
curl -X POST http://localhost:9200/api/response/blocklist \
429+
-H 'Content-Type: application/json' \
430+
-d '{"rule_type": "dst_ip", "pattern": "203.0.113.50", "description": "Known C2 server"}'
431+
432+
# Block a process chain pattern
433+
curl -X POST http://localhost:9200/api/response/blocklist \
434+
-H 'Content-Type: application/json' \
435+
-d '{"rule_type": "chain_pattern", "pattern": "** > curl > sh", "description": "Pipe curl to shell"}'
436+
437+
# Allowlist a known-good connection with chain scope
438+
curl -X POST http://localhost:9200/api/response/allowlist \
439+
-H 'Content-Type: application/json' \
440+
-d '{"rule_type": "dst_ip", "pattern": "18.97.36.79", "chain_filter": "launchd > Claude", "description": "Claude → Anthropic API"}'
441+
```
442+
443+
Rule types: `dst_ip`, `dst_cidr`, `domain`, `process_name`, `file_path`, `chain_pattern`. See [Chain-Aware Allow/Block Rules](#chain-aware-allowblock-rules) for the full chain pattern syntax.
444+
445+
### What Happens When a Threat Is Detected
446+
447+
| Stage | Learning | Passive | Active |
448+
|-------|----------|---------|--------|
449+
| Fast-path blocklist match | Skipped | Alert only | **Block immediately** — CRITICAL finding + response action |
450+
| LLM severity verdict | Recorded to baseline | Finding + alert | Finding → blocklist → allowlist → baseline → policy → approval → action |
451+
| Response action execution | Never | Never | Executes (with approval gate for destructive actions unless `auto_respond` is enabled) |
364452

365453
---
366454

@@ -374,7 +462,7 @@ edr-graph/
374462
│ ├── collectors/ # 10+ platform-native event sources
375463
│ ├── normalizer/ # OCSF normalization (6 event types)
376464
│ ├── schema/ # Kuzu DDL, SQLite DDL, OCSF types
377-
│ ├── processor/ # Entity extraction → graph writes
465+
│ ├── processor/ # Entity extraction, fast-path enforcement → graph writes
378466
│ ├── graph/ # Attack chain queries
379467
│ ├── analyzer/ # LLM tool-use analyzer + preflight
380468
│ ├── analysis/ # Lightweight detectors (DGA, persistence)
@@ -384,7 +472,7 @@ edr-graph/
384472
│ ├── dashboard/ # FastAPI server + SPA frontend
385473
│ ├── platform/ # Tamper detection, Windows service
386474
│ └── tray/ # macOS menu bar integration
387-
├── tests/ # 40 test modules, ~500 tests
475+
├── tests/ # 42 test modules, ~550 tests
388476
├── config.yaml # Runtime configuration
389477
└── README.md
390478
```

agent/dashboard/server.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,12 @@ async def get_status():
107107
if sample.name == "edr_events_dropped_total":
108108
events_dropped += int(sample.value)
109109

110+
events_fast_blocked = 0
111+
for metric in metrics.events_fast_blocked.collect():
112+
for sample in metric.samples:
113+
if sample.name == "edr_events_fast_blocked_total":
114+
events_fast_blocked += int(sample.value)
115+
110116
return {
111117
"agent_status": "paused" if _state["paused"] else "running",
112118
"uptime_seconds": round(uptime, 1),
@@ -115,6 +121,7 @@ async def get_status():
115121
"events_dropped": events_dropped,
116122
"events_per_second": round(events_processed / max(uptime, 1), 1),
117123
"queue_depth": queue.count_unprocessed(),
124+
"events_fast_blocked": events_fast_blocked,
118125
}
119126

120127

@@ -882,6 +889,10 @@ async def add_blocklist_rule(body: dict):
882889
except Exception:
883890
logger.exception("Failed to add blocklist rule")
884891
raise HTTPException(500, "Failed to add blocklist rule") from None
892+
# Invalidate fast-path blocklist enforcer cache
893+
fb = _state.get("fast_blocklist")
894+
if fb:
895+
fb.invalidate()
885896
return {"status": "ok", "rule_id": rule_id}
886897

887898

@@ -893,6 +904,10 @@ async def delete_blocklist_rule(rule_id: int):
893904
raise HTTPException(503, "Blocklist not initialized")
894905
if not blocklist.remove_rule(rule_id):
895906
raise HTTPException(404, "Rule not found")
907+
# Invalidate fast-path blocklist enforcer cache
908+
fb = _state.get("fast_blocklist")
909+
if fb:
910+
fb.invalidate()
896911
return {"status": "ok"}
897912

898913

agent/dashboard/static/index.html

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -890,14 +890,15 @@
890890

891891
// ── Settings ───────────────────────────────────────────────────
892892
async function refreshSettings() {
893-
const [data, iocStats, modeData, baselineStats, allowlistData, blocklistData, netStatus] = await Promise.all([
893+
const [data, iocStats, modeData, baselineStats, allowlistData, blocklistData, netStatus, statusData] = await Promise.all([
894894
api('/api/settings'),
895895
api('/api/intel/ioc-stats'),
896896
api('/api/response/mode'),
897897
api('/api/response/baseline/stats'),
898898
api('/api/response/allowlist'),
899899
api('/api/response/blocklist'),
900900
api('/api/response/network-status'),
901+
api('/api/status'),
901902
]);
902903
if (!data || Object.keys(data).length === 0) { document.getElementById('settings-content').innerHTML = '<div class="empty-state">Settings not available</div>'; return; }
903904
const groups = {
@@ -940,6 +941,9 @@
940941
if (allowlistData && allowlistData.rules) {
941942
html += '<div style="color:var(--dim);font-size:13px;margin-top:4px">Allowlist: <strong style="color:var(--text)">' + allowlistData.rules.length + '</strong> rules</div>';
942943
}
944+
if (statusData && statusData.events_fast_blocked > 0) {
945+
html += '<div style="color:var(--dim);font-size:13px;margin-top:4px">Fast-blocked: <strong style="color:var(--critical)">' + fmtNum(statusData.events_fast_blocked) + '</strong> events</div>';
946+
}
943947
html += '<div style="margin-top:8px"><button class="btn btn-danger" style="font-size:11px" onclick="clearBaseline()">Clear Baseline</button></div>';
944948
html += '</div></div>';
945949

0 commit comments

Comments
 (0)