Enable tool streaming

[MichaelMilstead]

Problem:

Often, tools we register with Tambo are to allow the LLM to take an action. Sometimes this action involves updating some test or component state (as with tambo's Interactables ). Currently, even though the LLM streams the generation of tool call definitions one chunk at a time, Tambo waits for the entire tool call to be built before returning it to the client. For some scenarios, this waiting can cause the application to feel slow.

We should enable developers to mark tools as "streamable", and when tambo decides to use those tools we should stream their "current state" to the client as they are being built.

Proposed Solution:

• Update react sdk TamboTool interface and API's ComponentContextToolMetadata interfaces to include a field specifying whether the tool is "streamable", defaulting to false

• Update handling of LLM response chunks to parse partial toolcall and, if the given tool is "streamable", return the "fixed json" to the client

• Update react sdk to handle partial toolcalls for "streamable" tools. The tool should be called, but no tool response message should be sent back to Tambo until the full message has been streamed and the final toolcall has returned.

• Update the 'update_interactable_component' tool in the TamboInteractableProvider to be 'streamable'

Acceptance Criteria

• Tools can be marked as "streamable" and when they are used, they are called with every new chunk returned

• Interactable update_interactable_component tool is streamable, and updates to "interactable comoponents" can be streamed

• Documentation of tools is updated to describe how to use the new field during tool registration, and how it affects tool calling

• Tests covering tool streaming

[shivam096]

Hey @MichaelMilstead ,

Is this available ?