-
Notifications
You must be signed in to change notification settings - Fork 368
Custom/aliased OpenAI-compatible providers: attachment caps may degrade to text-only #2741
Copy link
Copy link
Open
Open
Copy link
Labels
area/providersFor features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)area/providers/openaiFor features/issues/fixes related to the usage of OpenAI modelsFor features/issues/fixes related to the usage of OpenAI modelskind/fixPR fixes a bug (maps to fix: commit prefix)PR fixes a bug (maps to fix: commit prefix)status/needs-designRequires architectural discussion or design reviewRequires architectural discussion or design review
Metadata
Metadata
Assignees
Labels
area/providersFor features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.)area/providers/openaiFor features/issues/fixes related to the usage of OpenAI modelsFor features/issues/fixes related to the usage of OpenAI modelskind/fixPR fixes a bug (maps to fix: commit prefix)PR fixes a bug (maps to fix: commit prefix)status/needs-designRequires architectural discussion or design reviewRequires architectural discussion or design review
Type
Fields
Give feedbackNo fields configured for Bug.
Problem
Providers like xAI, Mistral, Nebius, Ollama, and other custom OpenAI-compatible endpoints route through
pkg/model/provider/oaistreamwith whateverProviderstring is set in the model config (e.g."xai","mistral","ollama"). Themodelcapssystem looks these up in models.dev, but the provider IDs in models.dev may not match the provider strings used in docker-agent configs.For example:
provider: xai, model: grok-2-vision-1212produces a lookup for"xai/grok-2-vision-1212"provider: ollama, model: llavaproduces"ollama/llava"If models.dev doesn't have those exact keys, capability detection silently falls back to text-only and image/PDF attachments are dropped.
Additionally, user-configured aliases (via
display_model) are already handled correctly byModelCapsID()(which usesModel, notDisplayOrModel()), but the underlying provider identifier mapping to models.dev still depends on the config'sproviderfield matching models.dev exactly.Possible fixes
Context
Identified during review of PR #2738 (fix: pass fully-qualified provider/model ID to modelcaps.Load).