🛡️ AegisLogic (PCRA)
Persistent Cognitive Retrieval Architecture

---

A local-first, zero-cloud-dependency AI retrieval system that combines three concurrent data tracks — temporal session history (SQLite), causal graph traversal (KuzuDB), and semantic vector search (Qdrant) — fused via Reciprocal Rank Fusion (RRF) and powered by a locally-running Llama 3.1 8B model on Apple Silicon hardware.

Built for the domain of cyber threat intelligence (CTI) — a vertical requiring strict temporal awareness of breach timelines, multi-hop causal reasoning across MITRE ATT&CK relationships, and zero hallucination tolerance when attributing threat actors to malware and mitigations.

Architecture Overview

                            ┌─────────────────────────┐
                            │     User Query Input     │
                            └────────────┬────────────┘
                                         │
                            ┌────────────▼────────────┐
                            │  Query Planner / Router  │
                            └────────────┬────────────┘
                                         │
              ┌──────────────────────────┼──────────────────────────┐
              ▼                          ▼                          ▼
   ┌─────────────────────┐  ┌─────────────────────┐  ┌─────────────────────┐
   │   Hindsight Track   │  │     KAG Track        │  │     RAG Track       │
   │  (SQLite History)   │  │  (KuzuDB Graph)      │  │  (Qdrant Vectors)   │
   └─────────┬───────────┘  └─────────┬───────────┘  └─────────┬───────────┘
              │                        │                          │
              └──────────────────────┬─┘──────────────────────────┘
                                     │  asyncio.gather (parallel)
                            ┌────────▼────────────┐
                            │  RRF Fusion Engine   │
                            └────────┬────────────┘
                                     │
                            ┌────────▼────────────┐
                            │ Context Compression  │
                            └────────┬────────────┘
                                     │
                            ┌────────▼────────────┐
                            │  Llama 3.1 (Ollama)  │
                            └────────┬────────────┘
                                     │
                            ┌────────▼────────────┐
                            │  Grounded Response   │
                            └─────────────────────┘

What You Install (on your MacBook M4)

Component	Where It Runs	Install Method	Why
Docker Desktop	Host (macOS)	Download	Runs Qdrant + the Python app containers
Ollama	Host (macOS)	brew install ollama	Must run on host to leverage M4 Neural Engine / GPU acceleration — Docker can't access Apple's ANE
Git	Host (macOS)	Pre-installed on macOS	Version control

Everything else runs inside Docker — you don't install Python, pip packages, KuzuDB, Qdrant, or sentence-transformers on your Mac directly.

Memory Budget (16 GB Unified Memory)

Subsystem	Allocation	Technology
macOS System	~3.50 GB	Apple Silicon unified memory
Ollama LLM	~4.80 GB	Llama 3.1 8B Q4_K_M (4-bit quantized)
KuzuDB Graph	~200 MB	In-process columnar (inside Docker)
Qdrant Vectors	~150 MB	Docker container
SQLite Index	~50 MB	Disk-backed (inside Docker)
Python + LangGraph	~300 MB	Docker container
Available Headroom	~6.97 GB	Buffer for OS paging

Project Structure

PCRA/
├── src/                          # Application source code
│   ├── __init__.py
│   ├── main.py                   # CLI entry point — runs the full pipeline
│   ├── config.py                 # Environment config (Ollama, Qdrant, paths)
│   ├── state.py                  # AgentState TypedDict + QueryIntent Pydantic schema
│   ├── entity_resolver.py        # RapidFuzz fuzzy entity normalization
│   ├── graph.py                  # LangGraph state machine assembly
│   ├── workers/                  # Async retrieval workers (one per data track)
│   │   ├── __init__.py
│   │   ├── hindsight.py          # Track A — SQLite temporal session history
│   │   ├── kag.py                # Track B — KuzuDB Cypher graph traversal
│   │   └── rag.py                # Track C — Qdrant semantic vector search
│   └── nodes/                    # LangGraph pipeline nodes
│       ├── __init__.py
│       ├── retrieval.py          # Parallel retrieval orchestrator (asyncio.gather)
│       ├── fusion.py             # Reciprocal Rank Fusion (RRF) scoring
│       ├── compression.py        # Context compression (regex boilerplate stripping)
│       └── generation.py         # LLM generation via Ollama (Llama 3.1)
├── data/                         # Runtime data stores
│   ├── knowledge_base/           # MITRE ATT&CK PDFs and threat intelligence reports for RAG ingestion
│   ├── graph_schema/             # KuzuDB schema + seed data
│   │   └── seed.cypher           # Cypher seed script (threat actors, malware, mitigations)
│   └── sessions/                 # SQLite DB file (created at runtime)
├── eval/                         # Evaluation framework
│   ├── evaluation_dataset.json   # Cybersecurity test queries (RAGAS harness)
│   ├── run_ragas_groq.py         # RAGAS evaluation runner (Groq 70B judge)
│   ├── debug_relevancy.py        # AnswerRelevancy diagnostic probe
│   ├── check_noncommittal.py     # Noncommittal classifier debug script
│   ├── ragas_report_groq.json    # Full evaluation results (70B)
│   ├── ragas_noncommittal_penalty.md  # 📄 Technical write-up: metric design flaw
│   └── run_ragas.py              # Legacy evaluation runner
├── Dockerfile                    # Multi-stage Python build (slim, non-root)
├── docker-compose.yml            # 2 services: aegis-app + qdrant
├── .dockerignore                 # Excludes .git, docs, node_modules from build
├── .env.example                  # Environment variable template
├── .gitignore                    # Python, Docker, data, IDE exclusions
├── requirements.txt              # Pinned Python dependencies
├── report.md                     # Full system engineering specification
└── README.md                     # ← You are here

Quick Start

1. Prerequisites

# Install Ollama (runs on your Mac host for M4 GPU acceleration)
brew install ollama

# Pull the quantized Llama 3.1 model (~4.8 GB download, one-time)
ollama pull llama3.1:8b-instruct-q4_K_M

# Verify Ollama is running
ollama list

2. Clone & Configure

git clone https://github.com/HarshSharma0007/PCRA.git
cd PCRA

# Create your local environment config
cp .env.example .env

3. Launch with Docker

# Build and start all services (Qdrant + AegisLogic app)
docker compose up --build

# Or run in detached mode
docker compose up --build -d

4. Run a Query

# Via Docker
docker exec -it aegis-app python -m src.main "Which courses of action mitigate the attack pattern used by the Lazarus Group?"

# Or run locally (if you have Python 3.11+ installed)
pip install -r requirements.txt
python -m src.main "What tools does APT29 use for spearphishing, and are there any recent VCDB incidents involving phishing?"

5. Run Evaluation

docker exec -it aegis-app python -m eval.run_ragas_groq

Evaluation Highlights

The system was evaluated using the RAGAS framework with both Llama 3.1 8B (local) and Llama 3.3 70B (Groq API) as LLM judges.

Metric	8B Judge	70B Judge	Notes
Faithfulness	0.750	1.000 ✅	Perfect grounding — zero hallucination
Answer Relevancy	0.647	0.486 ⚠️	Impacted by noncommittal penalty (see below)
Context Recall	0.625	0.750	Improved with stronger judge

🔍 Key Finding: The RAGAS "Noncommittal Penalty"

During evaluation, two queries scored exactly 0.000 for Answer Relevancy despite having perfect Faithfulness. A source-level investigation revealed a hardcoded binary penalty in the RAGAS AnswerRelevancy scorer that zeroes out scores when the LLM includes honest caveats like "I couldn't find specific information about..."

The root cause was traced to this line in the RAGAS source:

score = cosine_sim.mean() * int(not all_noncommittal)

📄 Full technical write-up: eval/ragas_noncommittal_penalty.md
🔧 Debug probes: eval/debug_relevancy.py | eval/check_noncommittal.py

Research Foundation

This architecture is grounded in 7 peer-reviewed AI papers:

#	Paper	Venue	Used For
1	RAG for Knowledge-Intensive NLP Tasks	NeurIPS 2020	Core retrieval-augmented generation paradigm
2	Sufficient Context	ICLR 2025	Guided abstention when context is insufficient
3	KAG: Knowledge Augmented Generation	Ant Group 2024	Graph-based knowledge augmentation layer
4	Speculative RAG	arXiv 2024	Parallel draft-verify retrieval architecture
5	Agentic RAG Survey	arXiv 2026	Multi-loop agentic execution patterns
6	Hindsight Memory	arXiv 2025	Structured agent memory (retain, recall, reflect)
7	AgentHER	arXiv 2026	Hindsight experience replay for trajectory learning

Tech Stack

Layer	Technology	Role
Orchestration	LangGraph + asyncio	State machine + parallel async fan-out
LLM Inference	Ollama + Llama 3.1 8B Q4_K_M	Local generation on M4 hardware
Vector Store	Qdrant (Docker)	Semantic similarity search
Graph Database	KuzuDB (embedded)	Causal/relational path traversal
Relational Store	SQLite	Temporal session history
Embeddings	all-MiniLM-L6-v2	384-dim local sentence embeddings
Entity Resolution	RapidFuzz	Fuzzy matching to canonical forms
Evaluation	RAGAS	Faithfulness + context recall metrics
Containerization	Docker + Docker Compose	Reproducible deployment

Known Limitations

• Entity normalization relies on a static canonical list (future: auto-populate from KuzuDB nodes)
• RRF treats all three tracks with equal weight (future: learnable track weighting)
• No TLP (Traffic Light Protocol) classification layer (recommended for real CTI deployment)
• Hindsight Critic Node is architecturally defined but not yet wired as a conditional LangGraph edge

License

MIT