OllamaRAG

A command-line interface (CLI) tool that enhances Ollama's capabilities with Retrieval-Augmented Generation (RAG) using PostgreSQL for vector storage.

Features

• Interactive CLI with rich text formatting
• Support for multiple Ollama models
• Vector similarity search for context-aware responses
• Persistent conversation history
• Document knowledge base management
• Command system for model switching and context viewing

Prerequisites

• PostgreSQL with vector extension
• Ollama installed and running locally
• Python 3.11+
• PostgreSQL database with vector similarity search capabilities

Installation

1. Clone the repository
2. Create a virtual environment and activate it:

   python -m venv ../venv
   source ../bin/activate

3. Install dependencies:

   pip install -r requirements.txt

4. Set up the database following the Database Setup instructions below
5. Configure environment variables in db/.env:

   DB_NAME=rag
   DB_USER=rag_user
   DB_PASSWORD=rag_password
   DB_HOST=localhost
   DB_PORT=5432
   

Database Setup

Prerequisites

• PostgreSQL 15+ installed
• pgvector extension for vector similarity search
• Python libraries: psycopg2-binary, python-magic for file type detection

Database Creation

1. Install pgvector extension:

   sudo apt install postgresql-15-pgvector  # For Ubuntu/Debian

2. Create database and user:

   CREATE DATABASE rag;
   CREATE USER rag_user WITH PASSWORD 'rag_password';
   GRANT ALL PRIVILEGES ON DATABASE rag TO rag_user;

3. Connect to the database and set up extensions:

   \c rag
   CREATE EXTENSION IF NOT EXISTS vector;

4. Create the documents table:

   CREATE TABLE documents (
       id SERIAL PRIMARY KEY,
       content TEXT NOT NULL,
       embedding vector(768),
       metadata JSONB,
       collection TEXT,
       created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
   );
   
   CREATE INDEX documents_embedding_idx ON documents 
   USING ivfflat (embedding vector_cosine_ops)
   WITH (lists = 100);

Utility Scripts

The repository includes several utility scripts for managing documents:

• process_documents.py: Bulk import documents (PDF, DOCX, TXT, CSV) into the vector database

  python process_documents.py
  # Follow prompts to specify document path and collection

• search_documents.py: Search through embedded documents

  python search_documents.py
  # Enter search query and optionally specify collection

• inspect_db.py: View database structure and statistics

  python inspect_db.py

Document Collections

Documents can be organized into collections for better organization:

• Documents without a collection are stored in general storage
• Collections are automatically created when processing documents
• Search can be performed across all collections or within specific ones

Usage

1. Run the CLI:

   python rag_cli.py

2. Select an Ollama model from the available list
3. Available commands:
   • /add <text> - Add document to knowledge base
   • /model <name> - Switch models
   • /context - View current context documents
   • /history - View conversation history
   • /reset - Reset conversation history and context
   • /help - Show help message
   • \quit or \q - Exit application
   • Ctrl+C - Exit using keyboard interrupt

Architecture

• rag_cli.py: Main CLI interface with async input handling and command processing
• db/vector_store.py: PostgreSQL vector database integration for document storage and similarity search
• utils/ollama_client.py: Ollama API client wrapper for model interaction and embedding generation

Embedding Model

This project uses two different Ollama models for distinct purposes:

1. Text Embeddings (nomic-embed-text-v1.5-F16)

   • Used for converting text into high-dimensional vector representations
   • Generates 768-dimensional embeddings for both documents and queries
   • Optimized for semantic similarity search
   • F16 quantization for efficient memory usage
   • Called via Ollama's embeddings API endpoint

2. Language Model (User Selected)

   • Used for generating responses and understanding context
   • Can be any model available in your Ollama installation
   • Selected at runtime from the available models list
   • Receives context from vector similarity search
   • Maintains conversation history for coherent dialogue

How Embeddings Work

1. Document Processing

   • When a document is added, it's split into chunks
   • Each chunk is converted to a 768-dimensional vector using nomic-embed-text
   • Vectors are stored in PostgreSQL using the pgvector extension

2. Query Processing

   • User questions are converted to vectors using the same model
   • PostgreSQL performs similarity search using cosine similarity
   • Most similar document chunks are retrieved and included as context
   • Only chunks with similarity above 0.7 are included in the prompt

3. Response Generation

   • Retrieved context is formatted with similarity scores
   • Context is combined with conversation history
   • Selected language model generates a response incorporating the context

Database Schema

The system uses a PostgreSQL table with vector similarity search capabilities:

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT NOT NULL,
    embedding vector(768),
    metadata JSONB,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX documents_embedding_idx ON documents 
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

How It Works

1. When you add a document, the system:

   • Generates embeddings using the Ollama API
   • Stores the document content, embeddings, and metadata in PostgreSQL

2. When you ask a question:

   • Your query is converted to an embedding
   • Similar documents are retrieved using vector similarity search
   • Retrieved context is included in the prompt to Ollama
   • The model generates a response incorporating the relevant context

3. The conversation history is maintained in memory and can be:

   • Viewed using the /history command
   • Reset using the /reset command
   • Used to provide conversation context to the model

Dependencies

• psycopg2-binary: PostgreSQL database adapter
• ollama: Ollama API client
• python-dotenv: Environment variable management
• rich: Terminal formatting and styling
• prompt-toolkit: Interactive command-line interface
• asyncio: Asynchronous I/O support