[Feature] Add response_format support, fix max_completion_tokens for gpt-5.x/o-series, and add Anthropic structured output in fallback chain

[ankushchhabradelta4infotech-ai]

Summary

Several provider compatibility issues in @yourgpt/llm-sdk that prevent the fallback chain from working correctly with newer OpenAI models and structured output requests.

Problem / Use Case

We use createFallbackChain + createRuntime to automatically failover between providers. Three issues block this from working correctly in production.

Issues

1. Newer OpenAI models (gpt-5.x, o1, o3, o4-*) fail with max_tokens

These models require max_completion_tokens instead of max_tokens, and do not accept temperature. Sending max_tokens causes a 400 error:

400 Unsupported parameter: 'max_tokens' is not supported with this model.
Use 'max_completion_tokens' instead.

Fix — detect by model name pattern (no hardcoding needed):

const isNewGen = /o1|o3|o4|gpt-5/.test(modelId);
// use max_completion_tokens if isNewGen, otherwise max_tokens
// skip temperature if isNewGen

Needs to be applied in:

• src/adapters/openai.ts → complete() and stream()
• src/providers/openai/provider.ts → doGenerate() and doStream()

---

2. response_format / json_schema is never forwarded to the provider

ChatRequest.config only has temperature and maxTokens. There is no responseFormat field, so structured output requests silently return plain text instead of JSON.

This breaks all frontend calls that use response_format: { type: "json_schema", json_schema: ... } with OpenAI and Gemini models.

Fix — add responseFormat to ChatRequest.config:

config?: {
  temperature?: number;
  maxTokens?: number;
  responseFormat?: {
    type: 'text' | 'json_object' | 'json_schema';
    json_schema?: { name: string; strict?: boolean; schema: Record<string, unknown> };
  };
}

And forward it in:

• src/adapters/openai.ts → complete() and stream() → pass as response_format
• src/providers/openai/provider.ts → doGenerate() and doStream() → pass as response_format
• src/providers/google/provider.ts → doGenerate() and doStream() → pass as response_format
• src/core/generate-text.ts → forward responseFormat to doGenerate/doStream
• src/types/language-model.ts → add responseFormat to DoGenerateParams

---

3. Anthropic adapter does not support structured output (output_config.format)

Anthropic's API has its own structured output format — output_config.format with { type: "json_schema", schema: {...} } — which is completely different from OpenAI's response_format. The SDK's Anthropic adapter does not support this at all.

When fallback chain falls to Anthropic on a json_schema call, the response comes back as plain text instead of structured JSON.

Fix — add responseFormat support to AnthropicAdapter:

• Map config.responseFormat.json_schema.schema → output_config: { format: { type: "json_schema", schema: ... } } in buildRequestOptions()
• Only send output_config when responseFormat.type === "json_schema"

Anthropic SDK reference: messages.create({ ..., output_config: { format: { type: "json_schema", schema: {...} } } })

---

Before submitting:

• I've searched existing issues to make sure this isn't a duplicate
• I've read the documentation

[Sahil5963]

Shipped in @yourgpt/llm-sdk@2.5.1-beta.0 via PR #95 (merged to beta).

pnpm add @yourgpt/llm-sdk@beta

All three items addressed:

1. ✅ max_completion_tokens for o1/o3/o4/gpt-5 — auto-detected via regex; temperature silently dropped for these models. Helpers isOpenAIReasoningModel() + buildOpenAITokenParams() in adapters/base.ts.
2. ✅ responseFormat forwarded everywhere — added unified ResponseFormat type to RequestLLMConfig, ChatRequest.config, DoGenerateParams, GenerateTextParams. Wired into all adapters (OpenAI, Anthropic, Google, Azure, xAI, Together, Fireworks, OpenRouter, Ollama).
3. ✅ Anthropic output_config.format — translated via toAnthropicOutputConfig() with schema sanitization (strips numeric/length constraints Anthropic rejects, converts oneOf → anyOf, forces additionalProperties: false).

Bonus:

• Per-provider schema sanitizers for Gemini (strips oneOf/anyOf/$ref/pattern)
• supportsJsonMode capability flag now actually read (warning logged on mismatch)
• Demo route /chat/structured in examples/fallback-demo exercises the full OpenAI → Anthropic → Gemini fallback chain
• Docs page: https://github.com/YourGPT/copilot-sdk/blob/beta/apps/docs/content/docs/llm-sdk/structured-output.mdx

Overlap note re: #92 — your PR routes gpt-5/o-series through the Responses API (different approach to the same problem). Both work; happy to drop the max_completion_tokens change here if you'd rather merge #92's approach. Open to a quick discussion in #92.

Follow-up tracked separately:

• feat(llm-sdk): Anthropic forced-tool fallback for models without output_config #96 — Anthropic forced-tool fallback (Vertex AI + older Claude 3 models)
• RFC: split generateObject() / streamObject() from generateText() / streamText() (Vercel AI SDK parity) #97 — RFC for generateObject() / streamObject() API split (Vercel parity)

Closing this issue. Please reopen if anything regressed when you test in the dashboard.

cc @ankushchhabradelta4infotech-ai