Chill, I Ground! 🌶 ️
Provenance-first extractive RAG: retrieve documents, select answer-relevant passages, and return source excerpts with citations instead of freely rewriting the evidence.
Traditional RAG systems retrieve relevant documents and then allow an LLM to freely generate a response. Verbatim RAG reduces that generative surface by selecting and displaying passages from retrieved context.
Built-in verified extraction paths return evidence text from the supplied
source. This is a provenance guarantee, not a truth guarantee: retrieval may be
incomplete, a source may be wrong, and an extractor may choose an irrelevant or
incomplete passage. The default contextual template may also generate
presentation text around cited excerpts; use template_mode="static" when the
framing must be fixed and deterministic.
On the paper's 100-row ACL-Verbatim benchmark, the 150M-parameter ACL-specialized model achieved 53.6 micro Word-F1, compared with 48.7 for the strongest evaluated LLM extractor. In the generic v2 model-card evaluation, v2 achieved higher micro Word-F1 than the evaluated Zilliz Semantic Highlight and Provence baselines on ACL, RAGBench, Squeez, and QASPER slices. See the paper for the benchmark design and limitations.
The pipeline can also use local encoder models for retrieval and extraction plus
static rendering, without generative LLM API calls. With SPLADE and
ModelSpanExtractor, that configuration supports CPU execution after model
weights are available.
| Property | Built-in exact/static path | Outside the guarantee |
|---|---|---|
| Evidence text | Returned from retrieved source text | Custom/structured extractors must enforce their own contract |
| Rendering | Exact excerpts plus fixed transparent framing | Contextual mode can generate introductions, labels, and connective text |
| Citations | Source citations and highlights are returned | Repeated identical text can still make source-offset mapping ambiguous |
| Correctness | Provenance can be inspected | Source truth, retrieval recall, relevance, completeness, and entailment |
# Install the package
pip install verbatim-ragFor local development:
pip install -e packages/core/
pip install -e .If you only need the reusable verbatim core without the full RAG pipeline (no torch, transformers, or Milvus):
pip install verbatim-corefrom verbatim_core import VerbatimTransform
vt = VerbatimTransform()
response = vt.transform(
question="What is the main finding?",
context=[
{"content": "The study found that X leads to Y.", "title": "Paper A"},
{"content": "Results show Z is significant.", "title": "Paper B"},
],
)
print(response.answer)Dependencies: only openai, pydantic, rapidfuzz, and jinja2.
| Surface | Location | Responsibility |
|---|---|---|
verbatim-core |
This repository, packages/core/ |
Reusable question + context → evidence transform, validation, templates, citations |
verbatim-rag |
This repository, verbatim_rag/ |
Reference ingestion, indexing, retrieval, and orchestration pipeline |
| Research/training | KRLabsOrg/acl-verbatim |
Paper reproduction, v2 training, datasets, and canonical evaluation |
| Hosted client | KRLabsOrg/verbatim-client |
SDK and CLI for hosted Verbatim services |
| Agent adapters | KRLabsOrg/verbatim-mcp, KRLabsOrg/verbatim-skill |
Thin MCP and agent integrations |
from verbatim_rag import VerbatimIndex, VerbatimRAG
from verbatim_rag.ingestion import DocumentProcessor
from verbatim_rag.vector_stores import LocalMilvusStore
from verbatim_rag.embedding_providers import SpladeProvider
# Process documents with intelligent chunking
processor = DocumentProcessor()
# Process PDFs from URLs
document = processor.process_url(
url="https://aclanthology.org/2025.bionlp-share.8.pdf",
title="KR Labs at ArchEHR-QA 2025: A Verbatim Approach for Evidence-Based Question Answering",
metadata={"authors": ["Adam Kovacs", "Paul Schmitt", "Gabor Recski"]}
)
# Create embedding provider and vector store
sparse_provider = SpladeProvider(
model_name="opensearch-project/opensearch-neural-sparse-encoding-doc-v2-distill",
device="cpu"
)
vector_store = LocalMilvusStore(
db_path="./index.db",
collection_name="verbatim_rag",
enable_dense=False,
enable_sparse=True,
)
# Create index with providers
index = VerbatimIndex(
vector_store=vector_store,
sparse_provider=sparse_provider
)
index.add_documents([document])
# Then query the index
rag = VerbatimRAG(index)
response = rag.query("What is the main contribution of the paper?")
print(response.answer)Set your OpenAI API key before using the system:
export OPENAI_API_KEY=your_api_key_here- Document Processing: Documents are processed using docling for format conversion and chonkie for chunking
- Document Indexing: Documents are indexed using vector embeddings (both dense and sparse)
- Template Management: Response templates are created and stored for common question types
- Query Processing:
- Relevant documents are retrieved
- Key passages are extracted verbatim using either LLM-based or fine-tuned span extractors
- Responses are structured using templates
- Citations link back to source documents
The evidence excerpts remain inspectable source text. Retrieval, extraction quality, and any generated contextual framing remain separate concerns.
- VerbatimRAG (
verbatim_rag/core.py): Main orchestrator that coordinates document retrieval, span extraction, and response generation - VerbatimIndex (
verbatim_rag/index.py): Vector-based document indexing and retrieval - SpanExtractor (
verbatim_rag/extractors.py): Abstract interface for extracting relevant text spans from documents- LLMSpanExtractor: Uses OpenAI models to identify relevant spans
- ModelSpanExtractor: Uses fine-tuned BERT-based models for span classification
- DocumentProcessor (
verbatim_rag/ingestion/): Docling + Chonkie integration for intelligent document processing - Document (
verbatim_rag/document.py): Core document representation with metadata
- Documents are processed and chunked using docling and chonkie
- Documents are indexed using vector embeddings
- User queries retrieve relevant documents
- Span extractors identify verbatim passages that answer the question
- Response templates structure the final answer with citations
- Responses expose selected source text with document references; guarantee details depend on the extractor and template mode described above
The repository contains a FastAPI API and Vite/React development UI. They are
not included in the PyPI wheel and are not yet part of the same compatibility
gate as verbatim-core. Reproducible local-stack work is tracked in
#27, and the document
lifecycle contract is tracked in
#31.
KRLabsOrg/verbatim-rag-modern-bert-v2 is a 150M-parameter query-conditioned token classifier built on gte-reranker-modernbert-base. It supports up to 8,192 tokens and is trained on scientific papers, Wikipedia QA, financial tables, medical literature, legal contracts, product manuals, and code/tool output.
The linked model card reports higher micro Word-F1 than the evaluated Zilliz Semantic Highlight and Provence baselines on ACL, RAGBench, Squeez, and QASPER slices. These are extractor evaluations, not end-to-end hallucination rates.
ModelSpanExtractor defaults to this model:
from verbatim_rag.core import VerbatimRAG
from verbatim_rag.index import VerbatimIndex
from verbatim_rag.extractors import ModelSpanExtractor
from verbatim_rag.vector_stores import LocalMilvusStore
from verbatim_rag.embedding_providers import SpladeProvider
extractor = ModelSpanExtractor(
model_path="KRLabsOrg/verbatim-rag-modern-bert-v2", # default
threshold=0.2,
min_span_chars=30,
merge_gap_chars=20,
device=None, # auto-detects cuda, mps, cpu
)
sparse_provider = SpladeProvider(
model_name="opensearch-project/opensearch-neural-sparse-encoding-doc-v2-distill",
device="cpu"
)
vector_store = LocalMilvusStore(
db_path="./index.db",
collection_name="verbatim_rag",
enable_dense=False,
enable_sparse=True,
)
index = VerbatimIndex(vector_store=vector_store, sparse_provider=sparse_provider)
rag_system = VerbatimRAG(
index=index,
extractor=extractor,
template_mode="static", # no generated contextual framing
k=5,
)
response = rag_system.query("Main findings of the paper?")
print(response.answer)| Resource | Link |
|---|---|
| 114K ACL Anthology papers in structured Markdown | KRLabsOrg/acl-anthology-md |
| Approximately 195K silver-labelled canonical query-chunk rows | KRLabsOrg/verbatim-spans |
| Human-annotated ACL extraction benchmark | KRLabsOrg/acl-verbatim-spans |
| Training and evaluation pipeline | KRLabsOrg/acl-verbatim |
If you use Verbatim RAG or the extractive models in your research, please cite our papers:
@misc{Recski:2026,
title={ACL-Verbatim: hallucination-free question answering for research},
author={Gábor Recski and Szilveszter Tóth and Nadia Verdha and István Boros and Ádám Kovács},
year={2026},
eprint={2605.21102},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2605.21102},
}
@inproceedings{kovacs-etal-2025-kr,
title = "{KR} Labs at {A}rch{EHR}-{QA} 2025: A Verbatim Approach for Evidence-Based Question Answering",
author = "Kovacs, Adam and
Schmitt, Paul and
Recski, Gabor",
editor = "Soni, Sarvesh and
Demner-Fushman, Dina",
booktitle = "Proceedings of the 24th Workshop on Biomedical Language Processing (Shared Tasks)",
month = aug,
year = "2025",
address = "Vienna, Austria",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.bionlp-share.8/",
pages = "69--74",
ISBN = "979-8-89176-276-3",
abstract = "We present a lightweight, domain{-}agnostic verbatim pipeline for evidence{-}grounded question answering. Our pipeline operates in two steps: first, a sentence-level extractor flags relevant note sentences using either zero-shot LLM prompts or supervised ModernBERT classifiers. Next, an LLM drafts a question-specific template, which is filled verbatim with sentences from the extraction step. This prevents hallucinations and ensures traceability. In the ArchEHR{-}QA 2025 shared task, our system scored 42.01{\%}, ranking top{-}10 in core metrics and outperforming the organiser{'}s 70B{-}parameter Llama{-}3.3 baseline. We publicly release our code and inference scripts under an MIT license."
}