🎯 Can a RAG System Know When It's Unsure?

Evaluating & improving uncertainty calibration in retrieval-augmented LLMs

MSc Data Science thesis — measuring whether a RAG model's confidence can be trusted for risk-sensitive decisions, and how embedding quality shapes that reliability.

---

🧩 The Problem

Retrieval-Augmented Generation (RAG) improves factual grounding — but LLMs still produce confidently wrong answers. In risk-sensitive domains (finance, healthcare, supply chain), an overconfident hallucination is worse than an honest "I don't know."

This thesis asks: How does embedding quality influence retrieval performance and uncertainty calibration in RAG systems used for risk-sensitive decision support? And critically — when the model is confident, is it actually right?

---

🔬 What I Did

A full RAG evaluation pipeline over real financial filings (annual & quarterly reports from 13 semiconductor and logistics companies — Nvidia, ASML, TSMC, Micron, FedEx, DHL, …):

1. Ingest & chunk the corpus (PyMuPDF + text cleaning).
2. Embed with 5 models for a controlled comparison — OpenAI text-embedding-3-small/large, bge-base, e5-base, mpnet-base.
3. Index & retrieve with FAISS; score retrieval with Recall@K, MRR.
4. Generate answers grounded in retrieved context; grade correctness with an LLM-as-judge.
5. Measure calibration — confidence vs. correctness via ECE and Brier score.
6. Improve calibration post-hoc with isotonic regression on a held-out split.

Evaluation set: 100 QA pairs (fact-based + risk-sensitive) generated from the corpus.

---

📊 Results

1 · Embedding model comparison

Model	Recall@1	Recall@3	Recall@5	MRR
bge-base 🥇	0.46	0.64	0.77	0.569
e5-base	0.39	0.68	0.79	0.545
openai-small	0.40	0.58	0.68	0.503
openai-large	0.34	0.53	0.67	0.453
mpnet-base	0.28	0.57	0.66	0.422

Key finding: the open-source bge-base embedding outperformed both OpenAI models on this financial corpus (best MRR and Recall@1), with e5-base strongest at higher K. Embedding choice — not just model size or vendor — materially changes retrieval quality.

2 · Uncertainty calibration

• Answer accuracy (LLM-as-judge): 66% over 100 questions.
• The raw model was overconfident — overall ECE 0.169, Brier 0.228.
• Post-hoc isotonic regression cut ECE by ~32% (0.135 → 0.092 on the held-out test split) and improved the Brier score (0.198 → 0.183) — confidence scores that better reflect reality.

Takeaway: RAG accuracy alone is not enough. Calibrating confidence is what makes a system safe to act on in risk-sensitive settings — and it can be improved cheaply, post-hoc.

---

🗂️ Repository

├── rag_calibration_experiments.ipynb   # full pipeline: ingest → embed → retrieve → judge → calibrate
├── make_charts.py                      # regenerates the result charts
├── charts/                             # result visualizations
├── results/                            # metrics as CSV (retrieval, calibration, generation)
├── DATA.md                             # corpus description + how to reproduce
└── requirements.txt

The copyrighted corpus is not redistributed — see DATA.md to reproduce it.

---

🧰 Tech Stack

Python · OpenAI (embeddings + generation) · Sentence-Transformers (BGE / E5 / MPNet) · FAISS (vector search) · LangChain · scikit-learn (isotonic regression, Brier) · PyMuPDF · Gemini (cross-model answer generation) · pandas / matplotlib

---

📈 Metrics Reference

Metric	What it measures
Recall@K	Was a relevant document retrieved in the top-K?
MRR	How high did the first relevant document rank?
ECE (Expected Calibration Error)	Gap between stated confidence and actual accuracy
Brier Score	Accuracy of probabilistic confidence predictions
LLM-as-Judge	Semantic correctness grading of generated answers

---

Baba Fakruddin Shaik · MSc Data Science · Trustworthy AI

This repository accompanies ongoing thesis research and will be updated as the work progresses.