When using native structured output, the LLM sometimes fails to strictly follow the prompt and does not wrap the response in json code block markers. This causes the framework to throw an error:
Failed to parse native structured output as JSON: Failed to deserialize JSON to java.lang.Object
I would like to use the Middleware.onModelCall hook to intercept the LLM response and modify/replace the TextBlockDeltaEvent to manually append the missing json markers before parsing, but it seems this is currently not supported.
Expected behavior
Allow replacing or modifying TextBlockDeltaEvent within Middleware.onModelCall so that developers can programmatically fix malformed LLM responses before they are parsed.
Environment
AgentScope-Java Version: 2.0.0
When using native structured output, the LLM sometimes fails to strictly follow the prompt and does not wrap the response in json code block markers. This causes the framework to throw an error:
Failed to parse native structured output as JSON: Failed to deserialize JSON to java.lang.Object
I would like to use the Middleware.onModelCall hook to intercept the LLM response and modify/replace the TextBlockDeltaEvent to manually append the missing json markers before parsing, but it seems this is currently not supported.
Expected behavior
Allow replacing or modifying TextBlockDeltaEvent within Middleware.onModelCall so that developers can programmatically fix malformed LLM responses before they are parsed.
Environment
AgentScope-Java Version: 2.0.0