Rag Search

Using AI Runner with RAG (Retrieval-Augmented Generation)

AI Runner provides a Retrieval-Augmented Generation (RAG) system that allows you to enhance your AI's responses with knowledge from your own documents.

Note: For long-term memory and fact retention, see the Knowledge System documentation.

How RAG Works in AI Runner

AI Runner uses a keyword-based retrieval system to find relevant information in your documents. When you ask a question, the system:

1. Extracts keywords from your query
2. Searches the index for documents containing those keywords
3. Retrieves the most relevant passages
4. Uses those passages to generate a more informed response

RAG vs Knowledge System

AI Runner provides two complementary systems for enhancing LLM responses:

• RAG Search (this page): Retrieves relevant passages from ingested documents (PDFs, text files, etc.) to answer questions based on your document library
• Knowledge System: Stores specific facts learned from conversations in a database for long-term memory and personalization

Use RAG for document-based Q&A and the Knowledge System for remembering user-specific facts and preferences.

Performance Optimizations

RAG search in AI Runner has been optimized for performance:

• Caching of keyword extraction: Previously extracted keywords are cached to avoid redundant processing
• Throttled index refreshing: The index is only refreshed after a certain time interval (5 minutes) to avoid excessive processing during multiple searches
• Batched document processing: Documents are processed in batches for more efficient indexing
• Progress reporting: Long-running operations show progress indicators
• Performance timing: Key operations are timed to help identify bottlenecks

Using RAG with Your Documents

To use RAG with your own documents:

1. Add documents in the AI Runner interface
2. The system will automatically index them
3. When chatting with your AI, it will now use these documents to enhance its responses

Troubleshooting

If RAG search seems slow:

• Reduce the number of documents in your index
• Use smaller documents or break large documents into smaller chunks
• Ensure your documents are properly formatted
• Check the logs for any error messages related to indexing