Skip to content

ALPHAintelligenceBD/semantic-law-search

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Semantic Law Search

Complete semantic search platform over EUR-Lex Sector 3 (EU legislation) — RAG, ingestion pipeline, zero-hallucination source verification via Chrome extension, and agentic workflows for legal practice management.

License: MIT Python 3.11+ Status: Archived No Profiling


⚠️ Compliance — No Profiling

This system MUST NOT be used for automated profiling of individuals. Such use would constitute:

  • A direct breach of the EU AI Act (Article 5 — prohibited practices, including social scoring and certain real-time biometric profiling)
  • A breach of GDPR Article 22 (automated individual decision-making, including profiling)

The ingestion pipeline indexes EUR-Lex Sector 3 documents — that is, EU legislation (Regulations, Directives, Decisions). It is not personal data, and is not intended to be combined with personal data to profile identifiable persons.

Any application built on this codebase MUST verify that:

  1. It does not perform profiling, social scoring, or automated decision-making about identifiable individuals
  2. It does not combine the legislative corpus with personal data in ways that produce inferences about individuals' behavior, preferences, or characteristics
  3. It complies with the AI Act risk classification appropriate to its deployment

If you are uncertain whether your use case constitutes profiling, do not deploy this system. Consult qualified legal counsel.


What this is

Semantic Law Search is a legal practice platform built for lawyers who need AI assistance they can actually trust. It combines semantic retrieval over EUR-Lex Sector 3 legislation with agentic workflows for billing, accounting, and CRM — all built on a state-first architecture where the database is the source of truth and the LLM is just a tool the system calls when needed.

The repo is described as "complete" because it ships the full stack: the ingestion pipeline (for adding EUR-Lex documents), the RAG core, the Chrome extension for one-click source verification, and the agentic modules (BHR, accounting, CRM). A lawyer with a GPU and the dependencies installed can ingest legislation and start querying it.

The intended use case is helping lawyers find relevant case law and statutory text more quickly, then verify the AI's citations against the official EUR-Lex source in one click. Not for replacing lawyer judgment. Not for profiling. Not for automated legal decisions about individuals.

Built solo over ~2.5 months (Nov 2025 – Jan 2026) as the flagship platform of Privacce Labs LLC. Now archived. The architecture and component code live here as a record and reference.

Training & ingestion data scope

  • Source: EUR-Lex Sector 3 (Legislation only — Regulations, Directives, Decisions)
  • CELEX format: 3{YYYY}{TYPE}{NNNN} — e.g., 32016R0679 (GDPR), 32024R1689 (AI Act), 32022R2065 (DSA)
  • Not included: EUR-Lex Sector 6 (case law), Sector 1 (treaties), or any personal-data corpus
  • Not designed for: profiling, behavioral inference, or any data about identifiable individuals

Architecture

The differentiating insight is state-first: instead of letting an LLM orchestrate workflows (which drift, hallucinate, and lose context), the database is the master. State changes trigger actions. The LLM is invoked only when needed, with explicit context and explicit return values.

┌─────────────────────────────────────────────────────────────────┐
│                       Frontend (React/TS)                        │
│  ┌─────────────┐  ┌──────────────┐  ┌───────────────────────┐   │
│  │  Chat UI    │  │  Verify Btn  │  │  Approval Queues      │   │
│  └──────┬──────┘  └──────┬───────┘  └──────────┬────────────┘   │
└─────────┼────────────────┼─────────────────────┼────────────────┘
          │                │                     │
          ▼                ▼                     ▼
┌─────────────────────────────────────────────────────────────────┐
│                     Backend (FastAPI)                            │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌─────────┐ ┌────────┐ │
│  │ RAG core │ │ Checker  │ │   BHR    │ │ Account │ │  CRM   │ │
│  │ /search  │ │ /checker │ │  /bhr    │ │/account │ │ /crm   │ │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬────┘ └───┬────┘ │
│       │            │            │            │          │       │
│       └────────────┴────────────┴────────────┴──────────┘       │
│                              │                                   │
│  ┌───────────────────────────▼─────────────────────────────┐   │
│  │   Long-Term Context: Profile + Standing Orders + Facts    │   │
│  └─────────────────────────────────────────────────────────┘   │
└──────────────────────────┬──────────────────────────────────────┘
                           │
        ┌──────────────────┼──────────────────────────┐
        ▼                  ▼                          ▼
┌──────────────┐   ┌────────────────┐    ┌─────────────────────┐
│   ChromaDB   │   │    SQLite      │    │   llama-server      │
│ (embeddings) │   │ (state + audit)│    │ (Qwen3-VL via GGUF) │
└──────────────┘   └────────────────┘    └─────────────────────┘
                                                    │
                                                    ▼
                                          ┌────────────────────┐
                                          │  ColQwen + BGE-m3  │
                                          │  Whisper (GPU ASR) │
                                          └────────────────────┘

The Chrome extension (in modules/checker/extension/) makes citation verification one-click: an LLM response with [SOURCE:32016R0679] opens the GDPR document on EUR-Lex, finds the cited text, and highlights it.


Status

Archived January 2026. Single-developer project, ~2.5 months of work. Not actively maintained. Code is preserved here for reference and as part of the Privacce Labs portfolio.

What works

  • ✅ RAG over EUR-Lex with CELEX citation system ([SOURCE:celex_id] format)
  • ✅ One-click source verification — opens EUR-Lex tab, finds and highlights cited text via Chrome extension
  • ✅ Voice interface (Whisper ASR, GPU-accelerated)
  • ✅ Billable Hours Report (BHR) — Chrome extension tracks lawyer activity
  • ✅ Agentic accounting — auto-detect case closure, escalating payment reminders
  • ✅ Agentic CRM — health scores 0-100, proactive communication suggestions
  • ✅ Long-term context system — 3-tier memory (User Profile, Standing Orders, Global Facts)
  • ✅ Multi-query decomposition for long context handling
  • ✅ Human-in-the-loop approval queues for sensitive actions
  • ✅ Audit-ready state management

What's incomplete

  • ⚠️ The Checker v6 documentation flagged a ModuleNotFoundError for ingestion_qwen3.py due to a refactor — ingestion_qwen3.py has been relocated to backend/ingestion/ in this archive to fix the import path
  • ⚠️ Some test outputs reference SQLite databases (accounting.db, core.db) that are gitignored; tests need fixtures regenerated to run
  • ⚠️ Multiple legacy RAG iterations (legacy_iterations/legalrag-v1/, v2/, v3-final/) preserved but not maintained

What never shipped publicly

  • 🚧 Multi-tenant deployment (designed but not deployed beyond single-user)
  • 🚧 Production deployment automation
  • 🚧 Migration from local llama-server to a managed inference service (was deliberate — local-first by design)

Quick start

This is archived code, not a maintained product. To run locally:

# 1. Clone
git clone https://github.com/ALPHAintelligenceBD/corpus.git
cd corpus

# 2. Backend dependencies
python -m venv .venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows
pip install -r backend/requirements.txt  # (or per-module requirements*.txt)

# 3. Start llama-server with a Qwen3-VL GGUF (separately downloaded)
# llama-server -m /path/to/qwen3-vl.gguf -c 32768

# 4. Run the FastAPI backend
cd backend
python server.py

# 5. (Optional) Build the Chrome extension
cd ../modules/checker/extension
# Load unpacked in Chrome → chrome://extensions

Models are not committed (multi-GB). See module READMEs for which models each component expects.


Key decisions

These are the four architectural choices a senior engineer would actually want to discuss:

1. State-first, not LLM-first

Most "AI agent" frameworks (LangChain, CrewAI, AutoGPT) let the LLM orchestrate. That works in demos and fails in production: context drifts, state is lost between calls, and there's no audit trail. Corpus inverts this: SQLite/Chroma is the master state, the LLM is invoked at specific steps with explicit context, and every action is logged. The system doesn't drift because there's nowhere for it to drift to — state lives in the database.

2. Zero-hallucination via the Checker

For lawyers, "the LLM said GDPR Article 6 says X" is worthless without the lawyer being able to verify it in 5 seconds. The Checker (modules/checker/) parses [SOURCE:32016R0679] citations from LLM output, opens the EUR-Lex document in a new tab, runs window.find() for the cited chunk, and shows a green "verified" overlay if the text matches — or amber "could not verify" if it doesn't. One click. That's the entire "trust but verify" loop.

3. Local-first model serving

Corpus runs on llama-server (llama.cpp HTTP server) with quantized Qwen3-VL GGUFs and a BGE-m3 embedder, plus a CUDA-accelerated Whisper for ASR. No OpenAI API. Reasons: (1) lawyer data never leaves the device by default, which matters for compliance, (2) cost — local inference is free at the margin, (3) GPU-accelerated Whisper hits real-time transcription on consumer hardware. Trade-off: setup is heavier than calling an API.

4. Modular FastAPI routers

Each capability (BHR, accounting, CRM, checker, ASR) is a separate APIRouter mounted under its own prefix in backend/server.py. This keeps modules independently testable and lets future versions swap implementations without touching the core. The downside is some router-to-router state has to go through the shared SQLite — that's intentional, that's the state-first design.


Repository layout

corpus/
├── backend/                    Main FastAPI server + ingestion
│   ├── server.py               Entry point (formerly SLSMAIN_DESKTOP_V1_SPLIT_READY.py)
│   ├── app.py
│   ├── queryChecker.py
│   ├── recording.py            Voice recording utilities
│   └── ingestion/
│       └── ingestion_qwen3.py  Document ingestion pipeline
│
├── modules/                    Feature modules (each is a FastAPI router + helpers)
│   ├── checker/                Source verification (frontend + backend + Chrome extension + 29 tests)
│   ├── billable_hours/         BHR receipt agent + accounting routers
│   ├── document_similarity/    4 approaches to document similarity
│   ├── print_workflow/         Document generation/printing
│   ├── source_checker/         Standalone source verifier backend
│   ├── agentic_edits/          State-driven document edit pipeline
│   ├── long_term_context/      3-tier memory system
│   └── security/               Audit logging, access control
│
├── docs/
│   ├── checker.md              Full Checker v6 technical documentation
│   └── test_results.md
│
├── tests/
│   ├── billable_hours/         BHR test suite
│   └── (general tests)
│
└── legacy_iterations/          Earlier versions kept for reference
    ├── legalrag-v1/
    ├── legalrag-v2/
    ├── legalrag-v3-final/
    └── checker-package/        Older checker iteration

A note on naming

Some files use an SLS prefix (sls_multiquery_checker.py, sls_shortcontext_checker.py). The acronym is from an earlier internal naming and has been preserved because multiple modules import these files by name — renaming the prefix would cascade across imports without adding clarity. The main entry point SLSMAIN_DESKTOP_V1_SPLIT_READY.py had no Python imports referencing it (it's a script, not a module), so it has been renamed to backend/server.py for clarity. No other refactoring has been done; this is the code as it was at archive time.


Stack

Backend: Python 3.11+, FastAPI, Pydantic, ChromaDB, llama-cpp-python, asyncio Models: Qwen3-VL (GGUF, via llama-server), BGE-m3 embedder, Whisper large-v3 (GPU) Data: SQLite (state + audit), ChromaDB (embeddings) Frontend: React, TypeScript, Tailwind, Framer Motion, lucide-react, MUI Icons Extension: Chrome Manifest V3, vanilla JS, content scripts + service worker Tests: pytest (29 unit tests in modules/checker/tests/)


Project history

Phase Dates Output
RAG foundations Nov 2025 Multiple semantic search engines (V1, V2, V2-FINAL, V2-semantic)
First legal RAG Nov–Dec 2025 LegalRAG v1, v2, v3 (preserved in legacy_iterations/)
Platform layer Dec 2025 Print Workflow, Document Similarity, Long-Term Context
Agentic workflows Jan 2026 BHR, accounting, CRM, agentic edits
Checker (zero-hallucination) Jan 2026 Source verification frontend + backend + Chrome extension + 29 tests
Archived Jan 2026 Pivoted to other projects; code preserved here

License

MIT — see LICENSE.


Built by Jawad Bin Rashid. The state-first architecture insight from this project informs ongoing work in healthcare AI (CueKeep) and agentic systems.

About

Semantic search platform over EU and US legal corpus (EUR-Lex, eCFR, NY State Law): RAG pipeline, zero-hallucination source verification via Chrome extension, agentic workflows. State-first architecture.

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages