Skip to content

Issue with Bedrock and Claude Model: ValidationException on Blank Text Field #1678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
2 tasks done
celeriev opened this issue May 9, 2025 · 4 comments · May be fixed by #1710
Open
2 tasks done

Issue with Bedrock and Claude Model: ValidationException on Blank Text Field #1678

celeriev opened this issue May 9, 2025 · 4 comments · May be fixed by #1710
Assignees
@DouweM
Labels
bug Something isn't working

Comments

@celeriev
Copy link

celeriev commented May 9, 2025 •
edited
Loading

Initial Checks

Description

While using pydantic Bedrock with Claude Model. I often reach the following issue:

botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The text field in the ContentBlock object at messages.21.content.0 is blank. Add text to the text field, and try again.

It's not always messages 21, it can be another. But most of the time it happens after a number of tool calls, potentially unsuccessful.

Unfortunately, I'm not sure to understand how to best replicate this. The following snippet led me to face this error in 50% of cases, so it can be used for that. However, I'd need some help to create a more robust code snippet to replicate this error.

I believe we could solve this from ensuring content is not empty in https://github.com/pydantic/pydantic-ai/blob/240b0120bf542e8dbeeb52e7b0f4836f964ededa/pydantic_ai_slim/pydantic_ai/models/bedrock.py#L367C15-L367C28

Example Code

from pydantic_ai import Agent, ModelRetry
from random import randint
import asyncio

def get_random_number():
    """
    Return a random number using the randint
    """
    res = randint(0, 100)
    if res > 50:
        raise ModelRetry("Error: Random number is greater than 50")
    return res

agent = Agent(
    "bedrock:us.anthropic.claude-3-7-sonnet-20250219-v1:0",
    system_prompt="You are a helpful assistant that generates random numbers.",
    retries=3,
    tools=[get_random_number],
)

async def main():
    async with agent.iter(user_prompt="Generate 10 different random numbers and then create a summary of these numbers.") as result:
        async for node in result:
            if Agent.is_model_request_node(node):
                async with node.stream(result.ctx) as request_stream:
                    async for event in request_stream:
                        print(event)

if __name__ == "__main__":
    asyncio.run(main())

Python, Pydantic AI & LLM client version

pydantic-ai-slim==0.1.10
Python 3.10.13
@DouweM
Copy link
Contributor

DouweM commented May 12, 2025

@celeriev Since there's nothing in the code you've shared or PydanticAI that intentionally builds empty text parts, I'm guessing it's Bedrock/Claude itself that's generating them and sending them to us, we're sending them back as part of the chat history along with a tool call response, and then Bedrock is complaining about what's effectively its own output.

To know exactly where to modify the code, we'll need to know a bit more about the type of content block the empty text appears on. Can you please add Logfire (https://ai.pydantic.dev/logfire/) and share the JSON body of the request that results in the error response?

@DouweM DouweM self-assigned this May 12, 2025
@celeriev
Copy link
Author

celeriev commented May 12, 2025 •
edited
Loading

Thanks for your tips @DouweM. Here's what I got with the help of logfire:

Error:
botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The text field in the ContentBlock object at messages.25.content.0 is blank. Add text to the text field, and try again.

JSON body of the message.25:

{'role': 'assistant', 'content': '''




''', 'tool_calls': [{'id': 'tooluse_sq7kr3FWQcuEQSQyyUsehw', 'type': 'function', 'function': {'name': 'get_random_number', 'arguments': ''}}], 'gen_ai.message.index': 25, 'event.name': 'gen_ai.assistant.message'}

You guessed correctly, it seems like bedrock returns a message for a tool call with empty content, causing it to break.

Happy to contribute to fixing this by the way

Edit: Another example:
Error: botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The text field in the ContentBlock object at messages.17.content.0 is blank. Add text to the text field, and try again.

JSON Body of message.17

{'role': 'assistant', 'content': '''

''', 'tool_calls': [{'id': 'tooluse_rzDW-OHAQuivpgCZ2Q6-kw', 'type': 'function', 'function': {'name': 'get_random_number', 'arguments': ''}}], 'gen_ai.message.index': 17, 'event.name': 'gen_ai.assistant.message'}

@DouweM
Copy link
Contributor

DouweM commented May 13, 2025

@celeriev All right, looks like we need a check here for item.content being blank:

pydantic-ai/pydantic_ai_slim/pydantic_ai/models/bedrock.py

Line 421 in 240b012

content.append({'text': item.content})

Can you please try that out and submit a PR if it works?

@DouweM DouweM added bug Something isn't working and removed need confirmation labels May 13, 2025
@celeriev celeriev linked a pull request May 13, 2025 that will close this issue
Open
@celeriev
Copy link
Author

Hello @DouweM, it is ready for review. This solved the issue for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DouweM DouweM

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't send whitespace-only assistant message back to Bedrock celeriev/pydantic-ai
2 participants
@DouweM @celeriev