How to deal with documents remaining in the RAG of open web ui

[jzakotnik]

When a chat is deleted after the AI perspective in TruSpace was built, the document remains in the vector DB of Open WebUI. This seems to be an issue, so questions are

• Should we delete the documents somehow?
• If not, are they used for further processing?

Some links

• File deletion doesn't properly clean up database entries, causing issues with re-uploads open-webui/open-webui#7181
• https://medium.com/@richard.meyer596/multi-source-rag-with-hybrid-search-and-re-ranking-in-openwebui-8762f1bdc2c6

[jzakotnik]

Have a look @SamuelPull @maxjmohr

[maxjmohr]

Some of my thoughts below:

• Are we truly using a RAG system? Do we actually need it, even though we are only creating the prompts once?
  • PRO: Don't build RAG at all but stringify documents and pass in prompt
  • CON: probably won't work for large docs because the prompt will get too large
• Let's say we are using a RAG system. Should the user have the ability to create new perspectives at any time?
  • If yes -> makes sense to keep the vectors in RAG to not load & delete over & over again (or perhaps a time to live)
  • If no -> delete the vectors as they only add noise for the creation of new perspectives on other documents & thus make the RAG less performant
• I have not yet deep dived into how to delete the vectors from the RAG in OpenWebUI. I am only familiar with other RAG frameworks (e.g. from LLamaIndex), where you can definitely delete, update documents.

[jzakotnik]

I think we need the RAG. In early versions we had reasonably large documents (e.g. 50 pages) and the results of the LLM without RAG were really bad.