[BUG] AzureOpenAIClient incorrectly handles max_completion_tokens

[stephentoub]

Library name and version

Azure.AI.OpenAI 2.5.0-beta.1

Describe the bug

Making a request to gpt-5 with chat completion while setting MaxOutputTokenCount results in errors when using AzureOpenAIClient but works successfully using OpenAIClient against the same deployment. As shown in the repro below, the only thing that differs is how the client instance is constructed.

Expected behavior

AzureChatClient: Hi! How can I help you today?

ChatClient: Hi! How can I help you today?

Actual behavior

AzureChatClient: HTTP 400 (invalid_request_error: unsupported_parameter)
Parameter: max_tokens

Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

ChatClient: Hi! How can I help you today?

Reproduction Steps

#pragma warning disable OPENAI001 // Type is for evaluation purposes only and is subject to change or removal in future updates. Suppress this diagnostic to proceed.
using OpenAI;
using Azure.Identity;
using System.ClientModel.Primitives;
using Azure.AI.OpenAI;
using OpenAI.Chat;

string endpoint = ...; // ADD ENDPOINT HERE
string deployment = "gpt-5";

ChatClient cc1 = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential()).GetChatClient(deployment);

ChatClient cc2 = new OpenAIClient(new BearerTokenPolicy(new DefaultAzureCredential(), "https://ai.azure.com/.default"), new OpenAIClientOptions()
{
    Endpoint = new Uri($"{endpoint}/openai/v1"),
}).GetChatClient(deployment);

Test(cc1);
Test(cc2);

void Test(ChatClient cc)
{
    try
    {
        var result1 = cc.CompleteChat([ChatMessage.CreateUserMessage("Hello!")], new ChatCompletionOptions { MaxOutputTokenCount = 1024 });
        Console.WriteLine($"{cc.GetType().Name}: {result1.Value.Content[0].Text}");
    }
    catch (Exception ex)
    {
        Console.WriteLine($"{cc.GetType().Name}: {ex.Message}");
    }
    Console.WriteLine();
}

Environment

No response

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jpalvarezl @trrwilson.

[trrwilson]

This stems from a very unfortunate service API limitation where older, pre-o1 models only support the legacy max_tokens property while newer models only support max_completion_tokens. As a result, the caller needs to select which variant to use and the best the library can do is choose a default -- which ends up being the old one for backwards compatibility.

There's a .SetNewMaxCompletionTokensPropertyEnabled() extension method for ChatCompletionOptions that will toggle the serialization to use the newer property value and unblock the use of MaxOutputTokenCount with these newer models like gpt-5. That is, updating the call to the following should resolve the failure:

        ChatCompletionOptions chatOptions = new()
        {
            MaxOutputTokenCount = 1024,
        };

        // Experimental; this was always intended to be short-term
        #pragma warning disable AOAI001
        // This will make newer models work (and make older models break)
        chatOptions.SetNewMaxCompletionTokensPropertyEnabled();
        
        var result1 = cc.CompleteChat([ChatMessage.CreateUserMessage("Hello!")], chatOptions);

Notably, using OpenAIClient directly with the newer /openai/v1 routes will also bypass this issue since the backwards compatibility quirk isn't enabled by default.

[stephentoub]

Thanks.

This stems from a very unfortunate service API limitation where older, pre-o1 models only support the legacy max_tokens property while newer models only support max_completion_tokens

I'm really surprised we can't fix that in the service. There shouldn't be any ambiguity: if the old property comes in for a model that only supports the new property, rename it, and vice versa. This seems like it should be a small bug fix that will save a lot of people pain and confusion.

There's a .SetNewMaxCompletionTokensPropertyEnabled() extension method for ChatCompletionOptions that will toggle the serialization to use the newer property value

As long as we need a workaround, have you considered placing that workaround on the AzureOpenAIClient instance rather than on the ChatCompletionOptions (making it a setting applied for all use of the client instance)? Where it currently is breaks the abstraction: you can't use a ChatClient like any other because you need to have access to this SetNewMaxCompletionTokensPropertyEnabled that comes from a completely different assembly. Anyone building libraries that just works in terms of a ChatClient is unlikely to be referencing Azure.AI.OpenAI.