What happened?
I attempted to have the CLI analyze a local MP4 video file. The CLI successfully used ffprobe and ffmpeg to extract frames and audio. However, when it attempted to send the extracted audio.mp3 back to the model, the API returned a 400 error:
Unable to submit request because the mime_type: audio/mpeg is not supported in function_response.parts.
This caused the CLI to enter an infinite error loop where it repeatedly tried to submit the same unsupported payload, making the chat session unusable. I had to manually terminate the process.
bug_report.json
What did you expect to happen?
I expected the CLI to either:
Successfully process the video and audio if the API supports it.
Provide a graceful error message explaining that audio/mpeg is currently unsupported for multimodal function calls, without entering an infinite loop.
Skip the audio extraction if the model/API cannot ingest it via the tool response path.
Client information
- CLI Version: 0.37.1
- Git Commit: 3b2d4f1
- Session ID: e0861c85-aefb-49bd-84b7-857d198e1752
- Operating System: linux v22.22.0
- Sandbox Environment: no sandbox
- Model Version: auto-gemini-3
- Auth Type: oauth-personal
- Memory Usage: 406.0 MB
- Terminal Name: VTE(7600)
- Terminal Background: #2e2e33
- Kitty Keyboard Protocol: Unsupported
Login information
Google account
Anything else we need to know?
Video Source: The video was an 8-second MP4 generated by Veo 3.1.
Technical Note: The issue seems to be a discrepancy between the CLI's read_file tool capabilities (which extracts audio) and the current limitations of the Vertex AI / Gemini API regarding supported MIME types in function_response.parts.
UX Impact: The repeating error loop prevents any further interaction in the session.
What happened?
I attempted to have the CLI analyze a local MP4 video file. The CLI successfully used ffprobe and ffmpeg to extract frames and audio. However, when it attempted to send the extracted audio.mp3 back to the model, the API returned a 400 error:
Unable to submit request because the mime_type: audio/mpeg is not supported in function_response.parts.
This caused the CLI to enter an infinite error loop where it repeatedly tried to submit the same unsupported payload, making the chat session unusable. I had to manually terminate the process.
bug_report.json
What did you expect to happen?
I expected the CLI to either:
Successfully process the video and audio if the API supports it.
Provide a graceful error message explaining that audio/mpeg is currently unsupported for multimodal function calls, without entering an infinite loop.
Skip the audio extraction if the model/API cannot ingest it via the tool response path.
Client information
Login information
Google account
Anything else we need to know?
Video Source: The video was an 8-second MP4 generated by Veo 3.1.
Technical Note: The issue seems to be a discrepancy between the CLI's read_file tool capabilities (which extracts audio) and the current limitations of the Vertex AI / Gemini API regarding supported MIME types in function_response.parts.
UX Impact: The repeating error loop prevents any further interaction in the session.