Defer tools to frontend

[JonathanWebNL]

I'm using neuron ai for a chat client where some tools should be executed in the frontend. Currently the tool is executed sync with the stream. Is it possible to pass the tool trough to the output stream so it can be handled in the frontend?

Currently this is possible with the OpenAI Assistant API, but I want to create a provider agnostic client.

[ilvalerione]

The agent automatically recognize the tool call and process it on the fly: https://github.com/inspector-apm/neuron-ai/blob/main/src/Providers/OpenAI/HandleStream.php#L72

It should be added some sort of option to not move forward automatically, but return the ToolCallMessage as final response.

[JonathanWebNL]

This behavior could be added to the Tool either explicitly as an option or implicitly when no callback and __invoke are set.

A check should be added in HandleStream.php at line 32 to determine if the tool should be executed there. If not, the ToolCallMessage should be yielded instead of sent to the provider, and added to the ChatHistory.

This would allow executing the tool on the frontend and streaming the ToolCallResultMessage separately.

This approach should make it provider independent.

[ilvalerione]

It's like a human in the loop for Tools.

https://docs.neuron-ai.dev/human-in-the-loop