Anthropic Structured Tool Call Ports

[Und3rf10w]

Background

The Anthropic provider recently added support for combining tool calling with structured outputs in commit cf4e2a9 (#9793). This feature allows developers to use tools (like web search, weather lookup, etc.) alongside structured JSON output schemas, enabling multi-step workflows with structured final outputs.

However, this capability was ported to the Amazon Bedrock and Google Vertex Anthropic providers

Summary

This PR ports the structured tool call support from the Anthropic provider to both Amazon Bedrock and Google Vertex Anthropic providers.

Amazon Bedrock Provider Changes

Core Implementation:

• Removed the incorrect warning that prevented using tools with JSON response format (bedrock-chat-language-model.ts:119-128)
• Fixed tool choice to use { type: 'required' } instead of { type: 'tool', toolName: 'json' } when combining tools with structured outputs (bedrock-chat-language-model.ts:133-134)
• Added isJsonResponseFromTool parameter to mapBedrockFinishReason function to correctly map finish reason from tool_use to stop when the JSON tool is used (map-bedrock-finish-reason.ts)
• Updated both doGenerate and doStream methods to pass usesJsonResponseFromTool flag to the finish reason mapper

Testing:

• Replaced the old warning test with two comprehensive test suites:
  • json schema response format with json tool response - Tests when only the JSON tool is called
  • json schema response format with other tool response - Tests when regular tools are called alongside JSON schema
• Tests verify tool configuration, response format, finish reasons, and provider metadata

Examples:

• Added examples/ai-core/src/generate-text/bedrock-output-array-tools.ts
• Added examples/ai-core/src/stream-text/bedrock-output-array-tools.ts

Google Vertex Anthropic Provider Changes

Since this provider inherits from the Anthropic provider through AnthropicMessagesLanguageModel, it automatically gains the feature. Added:

Testing:

• Added test verifying the feature works correctly through inheritance (google-vertex-anthropic-provider.test.ts:181-198)

Examples:

• Added examples/ai-core/src/generate-text/google-vertex-anthropic-output-array-tools.ts
• Added examples/ai-core/src/stream-text/google-vertex-anthropic-output-array-tools.ts

Manual Verification

Run the following examples to verify the feature works end-to-end:

Amazon Bedrock:

# Generate text with tools and structured output
pnpm tsx examples/ai-core/src/generate-text/bedrock-output-array-tools.ts

# Stream text with tools and structured output
pnpm tsx examples/ai-core/src/stream-text/bedrock-output-array-tools.ts

Google Vertex Anthropic:

# Generate text with tools and structured output
pnpm tsx examples/ai-core/src/generate-text/google-vertex-anthropic-output-array-tools.ts

# Stream text with tools and structured output
pnpm tsx examples/ai-core/src/stream-text/google-vertex-anthropic-output-array-tools.ts

Expected behavior:

• The model can use the weather tool to fetch information
• The final output is a structured JSON array matching the schema
• No warnings about tools being ignored with JSON response format
• Finish reason is correctly set to stop when JSON tool is used, and tool-calls when regular tools are used

Checklist

• Tests have been added / updated (for bug fixes / features)
• Documentation has been added / updated (for bug fixes / features)
• A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
• I have reviewed this pull request (self-review)

Future Work

I need to actually test this, hence why it's a draft for now

Related Issues

• Related to Not able to use tools with strucutred output in anthropic provider #9789 (original issue for Anthropic provider)
  • Implements the same feature for Amazon Bedrock and Google Vertex Anthropic providers

[lgrammel]

wont this removal introduce bugs when text messages are generated before the tool call?

[Und3rf10w]

I added some tests to cover potential edge cases like this in 0f28874, but either I'm doing this wrong and it's not catching them, or it's being accounted for by below:

Here we're adding all text parts to the content, regardless of usesJsonResponseTool:

ai/packages/amazon-bedrock/src/bedrock-chat-language-model.ts

Lines 323 to 326 in 0f28874

	for (const part of response.output.message.content) {
	// text
	if (part.text) {
	content.push({ type: 'text', text: part.text });

Then, while processing tool calls:

• If it's the JSON response tool, it sets isJsonResponseFromTool = true and adds the tool input as text content
• Otherwise, it adds a regular tool-call

ai/packages/amazon-bedrock/src/bedrock-chat-language-model.ts

Lines 362 to 369 in 0f28874

	const isJsonTool = usesJsonResponseTool && part.toolUse.name === 'json';
	if (isJsonTool) {
	isJsonResponseFromTool = true;
	content.push({
	type: 'text',
	text: JSON.stringify(part.toolUse.input),
	});

This block only skips tool-specific events (not text events) when it's a JSON response tool:

ai/packages/amazon-bedrock/src/bedrock-chat-language-model.ts

Lines 712 to 719 in 0f28874

	// when a json response tool is used, we don't emit tool events
	if (!isJsonTool) {
	controller.enqueue({
	type: 'tool-input-start',
	id: toolUse.toolUseId!,
	toolName: toolUse.name!,
	});
	}

• The usesJsonResponseTool flag only affects:
  • Converting the JSON tool's output to text format
  • Changing the finish reason from tool-calls to stop
  • Adding metadata flags

Are there other potential edge cases I might be missing to cover, or would you prefer I rewrote this in a more readable fashion?

[Und3rf10w]

Although I've tested this live now finally and I'm having trouble getting a text part. I'll come back to this in a few

[Und3rf10w]

@lgrammel

I have added more test coverage just in case because I was a bit worried about this. I cannot for the life of me find a case where that happens with the code currently.

This test, in theory, should cover that case:

ai/packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts

Lines 1630 to 1703 in c439b76

	it('should stream text introduction before JSON-only response', async () => {
	setupMockEventStreamHandler();
	prepareChunksFixtureResponse('bedrock-json-only-text-first.1');
	const { stream } = await model.doStream({
	prompt: [
	{ role: 'user', content: [{ type: 'text', text: 'Return name data' }] },
	],
	responseFormat: {
	type: 'json',
	schema: {
	type: 'object',
	properties: { name: { type: 'string' } },
	required: ['name'],
	additionalProperties: false,
	$schema: 'http://json-schema.org/draft-07/schema#',
	},
	},
	includeRawChunks: false,
	});
	const result = await convertReadableStreamToArray(stream);
	// Should emit text events for "Here is your data:" followed by JSON output as text
	expect(result).toMatchInlineSnapshot(`
	[
	{
	"type": "stream-start",
	"warnings": [],
	},
	{
	"id": "0",
	"type": "text-start",
	},
	{
	"delta": "Here is your data:",
	"id": "0",
	"type": "text-delta",
	},
	{
	"id": "0",
	"type": "text-end",
	},
	{
	"id": "1",
	"type": "text-start",
	},
	{
	"delta": "{"name":"John"}",
	"id": "1",
	"type": "text-delta",
	},
	{
	"id": "1",
	"type": "text-end",
	},
	{
	"finishReason": "stop",
	"providerMetadata": {
	"bedrock": {
	"isJsonResponseFromTool": true,
	},
	},
	"type": "finish",
	"usage": {
	"cachedInputTokens": undefined,
	"inputTokens": 100,
	"outputTokens": 20,
	"totalTokens": 120,
	},
	},
	]
	`);
	});

If you have any suggestions on how else to account for that, happy to make adjustments.

Also if you have any guidance on how to back port this to v5, I would love to do that as well.

[lgrammel]

ideally introduce tests with fixtures and snapshots like we have in the anthropic provider

[Und3rf10w]

ideally introduce tests with fixtures and snapshots like we have in the anthropic provider

1dde227 moved existing tests to fixtures