Skip to content

[Feature]: Forward OpenAI-compatible ListModels metadata instead of discarding it at unmarshal #7037

Description

@lucacolombo97

Prerequisites

  • I have searched existing issues and discussions to avoid duplicates

Problem to solve

OpenAIProvider.ListModels unmarshals the provider's /v1/models response into OpenAIModel, which keeps six fields: id, object, owned_by, created, active, context_window. ToBifrostListModelsResponse then maps four of them onto schemas.Model. Everything else the provider returned is discarded at unmarshal, before any consumer can see it.

The OpenAI-compatible surface is used by far more than OpenAI. Aggregators, gateways, inference hosts and self-served endpoints all speak it, and many return substantially richer model objects on that endpoint: per-token or banded pricing, maximum output tokens, a description, capability flags (vision, reasoning, tool calling, image generation, structured output), retirement or deprecation dates, data-retention and training-use properties, serving region, whether the weights are open, and canonical or upstream names identifying the underlying model and the lab that trained it.

Who benefits: anyone building a product surface on top of Bifrost's /v1/models. The gap is widest for providers that front several upstreams, where the model id encodes a host or a region rather than a vendor, so a consumer cannot attribute, group or price a model from the id alone. In that case the provider's own metadata is the only source, and Bifrost holds it and throws it away.

Concretely, without these fields a consumer cannot:

  • attribute a model to the lab that produced it, or group two ids that are the same underlying model served by different upstreams
  • show a price, when the provider publishes one and Bifrost's own catalog has no row for that id
  • distinguish a non-chat model (image generation, transcription, embedding) from a chat model
  • know in advance that a model will be refused by a provider-side compliance rule, rather than discovering it as a 403 at request time
  • know a model is scheduled for retirement

The practical consequence is a second integration straight to the provider, duplicating a call Bifrost already makes and a response it already parses.

Proposed solution

Widen OpenAIModel with optional fields and map them in ToBifrostListModelsResponse onto schemas.Model, which already has homes for most of them:

  • pricing → Pricing. Where a provider returns tiered bands, take the lowest-threshold band.
  • max output tokens → MaxOutputTokens
  • description → Description
  • retirement date → IsDeprecated when past, or a new RetiresAt
  • capability flags, canonical and upstream names, data-retention properties, region, open-weights → AdditionalAttributes, which exists for exactly this

Unknown JSON keys remain ignored, so providers that return none of this are unaffected and no existing response shape changes.

The general principle: for an OpenAI-compatible provider, prefer forwarding what the provider actually said over normalising it away. AdditionalAttributes makes that cheap for anything without a first-class home.

Alternatives considered

  • Calling the provider's API directly, alongside Bifrost. Works, and is the workaround we are using, but it means holding the provider credential in a second place and maintaining an integration for a call Bifrost already makes.
  • Bifrost's own model catalog. /api/models/parameters and /api/models/details are datasheet-derived. Coverage is good for first-party model names and thin for provider-scoped, region-suffixed or aggregator ids, so it cannot fill the gap for exactly the providers where the gap matters.
  • Reading Bifrost's pricing tables directly from its store. Some of this data is there, but coupling a consumer to Bifrost's schema across migrations is not acceptable.
  • Inferring from the model id. Vendor and family can often be guessed from name patterns, and we do today, but it breaks whenever a path segment names a host rather than a vendor, and it cannot produce prices, retention, retirement or capabilities at all.

Area(s)

Core (Go)

Additional context

The request extends work already in progress rather than opening a new direction: v2.1.0 added a custom UnmarshalJSON on OpenAIListModelsResponse accepting bare top-level arrays, using an inline struct to pick up organization and context_length for that form. Widening the envelope struct is the same class of change.

Relevant source:

  • core/providers/openai/types.go, OpenAIModel (line ~1007)
  • core/providers/openai/models.go, ToBifrostListModelsResponse
  • core/providers/openai/openai.go, ListModels

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions