docker

haiku.rag Docker Compose Example

Run haiku.rag with docling-serve for remote document processing, continuous ingestion via haiku-ingester, and a read-only MCP server.

Architecture

LanceDB allows exactly one writer + N readers per database URI, so the example runs the ingester and the MCP server as two separate containers sharing the same data volume:

• docling-serve-1 / docling-serve-2 - Two replicas of the document conversion + chunking service. The ingester round-robins jobs across them; running two means convert work overlaps and one container restarting (e.g. for memory recycling) doesn't stall ingest. Bumping to N replicas is the same pattern — duplicate the service block and add the URL to providers.docling_serve.base_url.
• haiku-ingester - Long-lived writer. Watches /docs, ingests new and changed files, queues retries, exposes the control plane on port 8765.
• haiku-rag - Read-only MCP server on port 8001 for AI assistant integration. Cannot write to the database — the ingester owns writes.

Both haiku.* services share the same slim image (built once) and the same config file; docker-compose overrides the image's default command to give each container its role.

This setup showcases the minimal haiku.rag-slim image combined with external document processing, ideal for production deployments.

Quick Start

# Create required directories
mkdir -p data docs

# Create config file from example (required)
cp haiku.rag.yaml.example haiku.rag.yaml

# Start services (pulls ghcr.io/ggozad/haiku.rag-slim:latest)
docker compose up -d

Place documents in docs/ for automatic indexing.

Building locally for development

The example pulls the published image. To run a local build of haiku.rag-slim instead — typical when iterating on the codebase — drop a docker-compose.override.yml next to docker-compose.yml (the file is auto-loaded by Compose and not checked in):

services:
  haiku-ingester:
    build:
      context: ../..
      dockerfile: docker/Dockerfile.slim
  haiku-rag:
    build:
      context: ../..
      dockerfile: docker/Dockerfile.slim

Then:

docker compose build         # builds & tags as ghcr.io/ggozad/haiku.rag-slim:latest
docker compose up -d         # uses the local image
docker compose pull          # back to the published image when done

Volume Mounts

Host Path	Container Path	Mounted on	Purpose
./data	/data	both haiku containers	Persistent LanceDB + ingester queue
./docs	/docs	haiku-ingester only	Documents to ingest (watched by the FS source)
./haiku.rag.yaml	/app/haiku.rag.yaml	both haiku containers	Configuration file

Important: The haiku.rag.yaml config file must exist before running docker compose up. Copy it from the example:

cp haiku.rag.yaml.example haiku.rag.yaml

The example config sets ingester.sources[0].root: /docs - this is the container path, not your host path. Documents placed in ./docs on your host will appear at /docs inside the container.

Usage

Add documents by dropping files into ./docs/ on the host — the ingester picks them up automatically (watchfiles + periodic sweep).

The haiku-rag container runs in read-only mode, so use it for queries:

# List documents
docker compose exec haiku-rag haiku-rag list

# Search
docker compose exec haiku-rag haiku-rag search "your query"

# Ask questions
docker compose exec haiku-rag haiku-rag ask "What is haiku.rag?"

Check ingester progress via its control plane:

curl http://localhost:8765/health
curl http://localhost:8765/jobs?status=queued
curl http://localhost:8765/dlq

Ports

• 5001 - docling-serve replica 1 API (with UI enabled, debug only)
• 5002 - docling-serve replica 2 API (host port; container still listens on 5001)
• 8001 - MCP server (read-only)
• 8765 - ingester control plane (/health, /jobs, /sources, /dlq)

Configuration

The setup uses haiku.rag-slim image configured to use docling-serve for document processing:

processing:
  converter: docling-serve
  chunker: docling-serve

providers:
  docling_serve:
    base_url: http://docling-serve:5001

Edit haiku.rag.yaml to configure providers, embeddings, and other settings. See the Configuration documentation for all options.

For API keys (OpenAI, Anthropic, etc.), set them as environment variables:

export OPENAI_API_KEY=your-key-here
export ANTHROPIC_API_KEY=your-key-here
docker compose up -d

The ingester container binds the control plane to 0.0.0.0 so the host port-mapping works. The example config requires a bearer token via INGESTER_TOKEN; set it in .env (gitignored) alongside the API keys before bringing the stack up:

echo "INGESTER_TOKEN=$(openssl rand -hex 32)" >> .env

Using a database server for the queue

By default the queue is a SQLite file on the ./data volume. To run it on Postgres instead, point ingester.queue.dburi at the server in haiku.rag.yaml:

ingester:
  queue:
    dburi: postgresql+asyncpg://haiku:secret@postgres:5432/haiku_rag

Add a Postgres service and wire the ingester to it with a docker-compose.override.yml (auto-loaded by Compose):

services:
  postgres:
    image: postgres:16-alpine
    environment:
      - POSTGRES_USER=haiku
      - POSTGRES_PASSWORD=secret
      - POSTGRES_DB=haiku_rag
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-U", "haiku", "-d", "haiku_rag"]
      interval: 5s
      timeout: 3s
      retries: 12
    restart: unless-stopped

  haiku-ingester:
    depends_on:
      postgres:
        condition: service_healthy

Workers claim jobs with FOR UPDATE SKIP LOCKED, so the ingester can run as several replicas against one Postgres queue to scale ingestion out. The LanceDB single-writer rule still holds, so multiple writers need LanceDB Cloud or another shared store rather than the local file volume.

Documentation

• Remote Processing
• Configuration
• CLI Commands
• MCP Server