[FEAT]: Support for CAG and KAG in AnythingLLM

[colegero]

What would you like to see?

Hello Team,

Thank you for your amazing work on AnythingLLM!

I would like to request the addition of support for Cache-Augmented Generation (CAG) and Key-Value Augmented Generation (KAG) in the platform.

Why this is important:
CAG: Preloading external knowledge into the model’s context window enables efficient responses without the need for real-time retrieval, which is useful for static or semi-static datasets.
KAG: Utilizing key-value caching can significantly speed up intermediate computations, especially for long-form content, reducing resource usage during generation.
Benefits for AnythingLLM:
Adding these features would:

Enhance response efficiency for users dealing with large datasets.
Reduce latency in query processing.
Expand the versatility of AnythingLLM for both dynamic and static use cases.
If this is already being considered, I would love to hear more. If not, I believe this enhancement could make a big difference for the community.

Thank you for considering this request, and keep up the excellent work!