Detection Explorer

Explore, compare, and track open-source detection rules across multiple security vendors — in one place.

🌐 Live Site: detectionexplorer.io

---

Detection Explorer ingests and normalizes detection rules from 11 major open-source security content repositories into a unified schema, enabling cross-vendor comparison, MITRE ATT&CK coverage analysis, and coverage gap identification.

Why?

Detection engineers work across multiple rule formats daily — Sigma YAML, Elastic TOML, Splunk YAML — each with different schemas, severity levels, and metadata structures. Detection Explorer normalizes all of them into a single searchable interface so you can:

• Search & filter across 11 vendors with full-text search, severity, status, and MITRE tactic/technique filters
• Compare coverage across vendors for any MITRE technique or keyword
• Identify gaps — find techniques covered by one vendor but missing from another
• Stay current — sync and re-ingest to pull the latest rules from each repo
• Export filtered results as JSON or CSV for downstream use

Supported Sources

Repository	Format
SigmaHQ	YAML
Elastic Detection Rules	TOML
Elastic Hunting Queries	TOML
Elastic Protections	TOML
Splunk Security Content	YAML
Sublime Rules	YAML
LOLRMM	YAML
Microsoft Sentinel	YAML
Google SecOps	YARA-L
Okta customer-detections	YAML (OIE / SPL)
Auth0 customer-detections	YAML (Sigma + SPL)

---

Development

Everything below is for running Detection Explorer locally.

Quick Start

# Clone the repository
git clone git@github.com:InfoSecJay/threat-detection-explorer.git
cd threat-detection-explorer

# Backend setup
cd backend
python -m venv venv
venv\Scripts\activate  # Windows
# source venv/bin/activate  # macOS/Linux
pip install -r requirements.txt

# Frontend setup (in a new terminal)
cd frontend
npm install

# Run (in separate terminals)
# Terminal 1 - Backend:
cd backend && python run.py

# Terminal 2 - Frontend:
cd frontend && npm run dev

The frontend will be available at http://localhost:5173 and the API at http://localhost:8000.

Prerequisites

• Python 3.11+
• Node.js 18+
• Git

Architecture

threat_detection_explorer/
├── backend/          # FastAPI Python backend
│   ├── app/
│   │   ├── api/      # REST API routes
│   │   ├── models/   # SQLAlchemy database models
│   │   ├── parsers/  # Vendor-specific rule parsers
│   │   ├── normalizers/  # Rule normalization logic
│   │   └── services/ # Business logic services
│   └── tests/        # Pytest test suite
├── frontend/         # React TypeScript frontend
│   └── src/
│       ├── components/
│       ├── pages/
│       ├── hooks/
│       └── services/
└── data/            # Runtime data (git-ignored)
    ├── repos/       # Cloned repositories
    └── threat_detection.db  # SQLite database

Usage

1. Sync Repositories

On first use, sync the detection rule repositories:

1. Go to the Dashboard
2. Click "Sync" for each repository (SigmaHQ, Elastic, Splunk, etc.)
3. Wait for the clone/pull to complete

2. Ingest Rules

After syncing, ingest the rules into the database:

1. Click "Ingest" for each synced repository
2. Wait for parsing and normalization to complete

3. Explore Detections

• Browse: Go to Detections page to search and filter rules
• Compare: Use the Compare page to see coverage across vendors
• Export: Download filtered results as JSON or CSV

API Endpoints

Health

• GET /api/health - Health check

Repositories

• GET /api/repositories - List all repositories
• POST /api/repositories/{name}/sync - Sync a repository
• POST /api/repositories/{name}/ingest - Ingest rules from repository

Detections

• GET /api/detections - List detections with filters
• GET /api/detections/{id} - Get detection details
• GET /api/detections/statistics - Get statistics

Compare

• GET /api/compare?technique=T1059 - Compare by MITRE technique
• GET /api/compare?keyword=powershell - Compare by keyword
• GET /api/compare/coverage-gap?base_source=sigma&compare_source=elastic - Find coverage gaps

Export

• POST /api/export - Export detections as JSON or CSV

Normalized Schema

Every rule from every source ends up looking the same — ~40 canonical fields covering identity, status/severity, the canonical platform/data-source/event-type taxonomy, MITRE mapping, and observables extracted from the detection logic itself.

Full reference: docs/schema.md — field-by-field table, per-source vendor → canonical mapping, and a worked round-trip example.

For depth on how the canonical taxonomy resolver works per vendor, see docs/taxonomy.md.

Running Tests

cd backend
pytest tests/ -v

Configuration

Environment variables (can be set in .env):

• DEBUG - Enable debug mode (default: false)
• DATABASE_URL - SQLite database URL (default: sqlite+aiosqlite:///./data/threat_detection.db)
• CORS_ORIGINS - Allowed CORS origins (default: http://localhost:5173,http://localhost:3000)

License

MIT