Async Chat with RAG #326
-
|
Hello, When extending the NeuronAI\RAG\RAG class and call chatAsync it doesn't use RAG. $agentOne = AgentOne::make();
$agentTwo = AgentTwo::make();
$results = Utils::unwrap([
'agentOne' => $agentOne->chatAsync(new Message(MessageRole::USER, $prompt)),
'agentTwo' => $agentTwo->chatAsync(new Message(MessageRole::USER, $prompt)),
]);In this case only AgentOne extends RAG, the other extends Agent. The response is faster with the two parallel calls but AgentOne doesn't use the configured vector store. I see the RAG class doesn't have a chatAsync method, is this intentional or a misc bug? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Supported in the latest release 2.5.5 It's not efficient as it is with simple agents, because the initial retrieval process must be executed outside of the async request, but it makes finally possible to parallelize the inference. |
Beta Was this translation helpful? Give feedback.
Supported in the latest release 2.5.5
It's not efficient as it is with simple agents, because the initial retrieval process must be executed outside of the async request, but it makes finally possible to parallelize the inference.