Skip to content

Inferred strict=true may cause compatibility issues with OpenAI-compatible servers #1561

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 tasks done
chizukicn opened this issue Apr 21, 2025 · 7 comments
Closed
2 tasks done
Assignees

Comments

@chizukicn
Copy link

chizukicn commented Apr 21, 2025

Initial Checks

Description

🐛 Bug Description

When defining tools with required parameters using the MCP Server. the pydantic-ai automatically infers and inserts "strict": true into the request, even if the original tool definition explicitly sets strict to null.

This behavior causes compatibility issues with some OpenAI-compatible servers that do not support the strict field, as the pydantic-ai automatically infers and adds strict: true.

🔍 Tool Registration JSON Example

Here is the actual JSON representation of a registered tool:

[
  {
    "name": "test",
    "description": "test required",
    "parameters_json_schema": {
      "type": "object",
      "properties": {
        "some": {
          "type": "string",
          "description": "anything"
        }
      },
      "required": ["some"],
      "additionalProperties": false,
      "$schema": "http://json-schema.org/draft-07/schema#"
    },
    "outer_typed_dict_key": null,
    "strict": null
  }
]

❌ Actual Request Payload

Despite the tool definition having strict as null, the request payload includes the inferred strict: true:

{
  "messages": [{ "role": "user", "content": "hello" }],
  "model": "qwen/qwq-32b",
  "n": 1,
  "stream": true,
  "stream_options": { "include_usage": true },
  "tool_choice": "auto",
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "test",
        "description": "test required",
        "parameters": {
          "type": "object",
          "properties": {
            "some": {
              "type": "string",
              "description": "anything"
            }
          },
          "required": ["some"],
          "additionalProperties": false
        },
        "strict": true  // ← Automatically inferred as true despite being null in the original definition
      }
    }
  ]
}

This request will cause compatibility issues with OpenAI-compatible servers that do not support the strict field.

Image

Example Code

import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";

import { z } from "zod";

// Create an MCP server
const server = new McpServer({
  name: "MCP Test",
  version: "1.0.0",
});

server.tool("test", "test required", {
  some: z.string().describe("anything")
}, async () => {
  return {
    content: []
  };
});

const transport = new StdioServerTransport();

server.connect(transport);

Python, Pydantic AI & LLM client version

python==3.13
pydantic-ai==0.1.3

Note: This issue was drafted and refined with the help of ChatGPT.

@DouweM
Copy link
Contributor

DouweM commented Apr 21, 2025

@chizukicn Thanks for raising this. This is really part of the bigger issue where "OpenAI-compatible" APIs are really only compatible with specific (older) versions of the OpenAI API, and even then they may behave differently on the details. Our OpenAIModel always targets the latest version of the OpenAI API, so since "compatible" APIs can't be expected to always be caught up, we'll keep running into issues like this.

It may be time to abstract OpenAIModel into something like an OpenAICompatibleModel that can have specific API features (like strict) turned turned off or configured differently. OpenAIModel would then inherit from this class and turn on all the features. We could define other named subclasses for well-known "compatible" APIs as well.

@dmontagu What do you think?

@DouweM
Copy link
Contributor

DouweM commented May 20, 2025

@chizukicn I'm gonna look into fix this -- just for completeness, which OpenAI-compatible API were you using here? I can see the model is Qwen but not the provider.

@chizukicn
Copy link
Author

@chizukicn I'm gonna look into fix this -- just for completeness, which OpenAI-compatible API were you using here? I can see the model is Qwen but not the provider.

Thanks for looking into this! I'm using the OpenAI-compatible API provided by a Chinese cloud service platform: ppinfra.com (API endpoint: https://api.ppinfra.com/v3/openai).
You can see the relevant documentation, though it appears to be available only in Chinese.
The link is: https://ppio.cn/docs/model/llm#%E4%BB%A3%E7%A0%81%E7%A4%BA%E4%BE%8B

@chizukicn
Copy link
Author

chizukicn commented May 23, 2025

@DouweM What I find a bit hard to understand is that, based on packet capture, when there is a required parameter in the MCP tool, it includes the argument strict=True; however, if the parameter is optional, the strict argument is not present.

@DouweM
Copy link
Contributor

DouweM commented May 23, 2025

@chizukicn See here: https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/models/openai.py#L1014. OpenAI only supports strict mode when there are no optional fields.

@DouweM DouweM closed this as completed May 26, 2025
@DouweM
Copy link
Contributor

DouweM commented May 26, 2025

@chizukicn With the changes in #1835, you should be able to stop PydanticAI from sending strict=true to your model like this:

model = OpenAIModel(model_name=..., provider=..., profile=OpenAIModelProfile(openai_supports_strict_tool_definition=False))

If you have a chance, could you verify that that works as expected?

We could also include a new PPInfraProvider that does that automatically, if you think they're significant enough in the Chinese market for that to be a timesaver for future users.

@chizukicn
Copy link
Author

@chizukicn With the changes in #1835, you should be able to stop PydanticAI from sending strict=true to your model like this:

model = OpenAIModel(model_name=..., provider=..., profile=OpenAIModelProfile(openai_supports_strict_tool_definition=False))
If you have a chance, could you verify that that works as expected?

We could also include a new PPInfraProvider that does that automatically, if you think they're significant enough in the Chinese market for that to be a timesaver for future users.

Nice! This problem is resloved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants