-
Notifications
You must be signed in to change notification settings - Fork 2.5k
Improvements to base openai compatible #9462
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Re-reviewed commit 3b468dd. ZAI refactoring looks good, but the previously flagged issue remains unaddressed.
Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues. |
| cacheWriteTokens: cacheWriteTokens || undefined, | ||
| cacheReadTokens: cacheReadTokens || undefined, | ||
| totalCost, | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new processUsageMetrics method in the base class now includes cost calculation via calculateApiCostOpenAI. However, FeatherlessHandler has a custom createMessage implementation for DeepSeek-R1 models that yields usage without cost calculation (lines 79-85). This creates an inconsistency: non-R1 models (which use super.createMessage()) will get cost calculation, but R1 models won't. Consider updating the R1 path to also use processUsageMetrics or call calculateApiCostOpenAI directly to maintain consistent cost tracking across all Featherless models.
Fix it with Roo Code or mention @roomote and request a fix.
Ideally we can share more between providers - I think this works for Groq and Baseten at least.
Important
Enhance base OpenAI-compatible provider for better compatibility and flexibility across different providers, with updates to test assertions and usage metrics handling.
toMatchObjectfor partial matching inbase-openai-compatible-provider.spec.ts,featherless.spec.ts,fireworks.spec.ts,groq.spec.ts,io-intelligence.spec.ts,sambanova.spec.ts, andzai.spec.ts.createMessageandcompletePromptmethods inbase-openai-compatible-provider.tsto handle reasoning content and usage metrics more flexibly.processUsageMetricsmethod inbase-openai-compatible-provider.tsto calculate and yield usage metrics.createStreamandcreateMessagemethods to include reasoning parameters if supported by the model.groq.tsandzai.tsby leveraging the base provider's functionality.cacheWriteTokensandcacheReadTokensare undefined when zero ingroq.spec.tsandzai.spec.ts.This description was created by
for 3b468dd. You can customize this summary. It will automatically update as commits are pushed.