added more model versions #5293

chitalian · 2025-11-25T01:19:31Z

Ticket

Link to the ticket(s) this pull request addresses.

Component/Service

What part of Helicone does this affect?

Type of Change

Testing

Technical Considerations

Database migrations included (if needed)
API changes documented
Breaking changes noted
Performance impact assessed
Security implications reviewed

Dependencies

No external dependencies added
Dependencies added and documented
Environment variables added/modified

Deployment Notes

No special deployment steps required
Database migrations need to run
Environment variable changes required
Coordination with other teams needed

Context

Why are you making this change?

Screenshots / Demos

Before	After

Misc. Review Notes

vercel · 2025-11-25T01:19:37Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
helicone	Ready	Preview	Comment	Nov 25, 2025 5:38am
helicone-bifrost	Ready	Preview	Comment	Nov 25, 2025 5:38am
helicone-eu	Ready	Preview	Comment	Nov 25, 2025 5:38am

claude · 2025-11-25T01:19:52Z

Claude finished @chitalian's task —— View job

Code Review Complete

Todo List:

Read and analyze changed files to understand model additions
Review code quality, patterns, and potential issues
Check for bugs, security issues, and performance concerns
Provide detailed feedback with specific recommendations
Give final confidence score and summary

Review Analysis

This PR adds comprehensive support for new OpenAI model versions across GPT-4o, GPT-4.1, GPT-5, and GPT-5.1 families. The implementation follows Helicone's established patterns for model versioning with dated snapshots.

Critical Issues Found

🚨 Future Creation Dates

Several models use future creation dates (2025-03-11, 2025-04-14, 2025-08-07, 2025-11-13)
This could cause analytics/billing issues for date-based filtering and reporting
Location: packages/cost/models/authors/openai/gpt-4o/models.ts:44, gpt-4.1/models.ts:11, gpt-5/models.ts:75, gpt-5.1/models.ts:63

⚠️ Pricing Inconsistencies

Inconsistent web search pricing: GPT-4o/4.1/5/5.1 use $20/1K searches (0.02) while other OpenAI models use $10/1K (0.01)
This creates billing inconsistency across the model catalog
Location: All new endpoint files vs existing models like o1, o3, oss

⚠️ Model Output Capabilities Inconsistency

GPT-5.1 models support outputs: ["text", "image"] while all other GPT models only support outputs: ["text"]
This inconsistency could confuse users about actual model capabilities
Location: packages/cost/models/authors/openai/gpt-5.1/models.ts:12,22,32,43,64

Code Quality Issues

Style Consistency

Model naming: GPT-5.1 dated version has identical name to base model, making them indistinguishable in UI
Location: gpt-5.1/models.ts:57 - Consider adding "(2025-11-13)" suffix

Parameter Support Variations

GPT-5.1 models use unsupportedParameters array while others don't explicitly list unsupported params
Helicone provider endpoints have reduced parameter support compared to OpenAI/Azure/OpenRouter
This follows existing patterns but could be documented better

Security & Performance

✅ Security: No security issues found - all pricing and configuration data is static
✅ Performance: Changes only add model configurations, no runtime performance impact
✅ Breaking Changes: None - purely additive changes maintaining backward compatibility

Positive Aspects

Excellent pattern consistency with existing model structure
Comprehensive provider support (OpenAI, Azure, OpenRouter, Helicone)
Proper use of pinnedVersionOfModel for version mapping
Consistent pricing structure across providers with OpenRouter markup
All new models include proper context lengths, rate limits, and supported parameters

Score: 6/10 - Functional implementation with several consistency issues requiring attention

Suggestions Summary:

Critical: Fix future creation dates to prevent analytics issues
Important: Standardize web search pricing across all OpenAI models ($10 vs $20/1K)
Important: Align GPT-5.1 output capabilities with other GPT models
Style: Add version suffixes to dated model names for UI clarity
Documentation: Consider documenting parameter support variations

greptile-apps · 2025-11-25T01:25:27Z

Greptile Overview

Greptile Summary

This PR adds support for new OpenAI model versions across multiple GPT families in Helicone's cost calculation system. The changes introduce dated model variants including GPT-4o search preview models (2025-03-11), GPT-4.1 versions (2025-04-14 and 2024-07-18), GPT-5 family snapshots (2025-08-07), and GPT-5.1 variant (2025-11-13). These additions follow Helicone's established pattern for model versioning, where specific dated versions are created as immutable snapshots while maintaining references to their base models through pinnedVersionOfModel properties.

The model configurations are comprehensive, spanning multiple providers (OpenAI, Azure, OpenRouter, Helicone) with consistent pricing, rate limits, context lengths, and parameter support. Notable additions include search-enabled GPT-4o variants with web search capabilities priced at $10 per1000 searches. This change enables accurate cost tracking and billing for users who reference specific model versions rather than the base model names, supporting both compatibility and reproducibility in LLM applications.

Important Files Changed

Filename	Score	Overview
packages/cost/models/authors/openai/gpt-4o/endpoints.ts	5/5	Added search-preview model configurations with web search capabilities across all providers
packages/cost/models/authors/openai/gpt-4o/models.ts	4/5	Added model definitions for search-enabled GPT-4o variants with enhanced web search capabilities
packages/cost/models/authors/openai/gpt-4.1/endpoints.ts	4/5	Added endpoint configurations for two new dated GPT-4.1 model versions across multiple providers
packages/cost/models/authors/openai/gpt-4.1/models.ts	5/5	Added model registry entries for dated GPT-4.1 variants with proper version mapping
packages/cost/models/authors/openai/gpt-5/endpoints.ts	5/5	Added comprehensive endpoint configurations for four new dated GPT-5 model variants
packages/cost/models/authors/openai/gpt-5/models.ts	4/5	Added model definitions for dated GPT-5variants with consistent version referencing
packages/cost/models/authors/openai/gpt-5.1/endpoints.ts	4/5	Added endpoint configuration for new GPT-5.1 variant with provider-specific variations
packages/cost/models/authors/openai/gpt-5.1/models.ts	4/5	Added model definition for GPT-5.1 variant with future creation date

Confidence score: 4/5

This PR is safe to merge with minimal risk as it only adds new model configurations without modifying existing functionality
Score reflects comprehensive model additions that follow established patterns, though some models use future creation dates that may cause temporal inconsistencies in analytics
Pay close attention to the Helicone provider configurations which have reduced capabilities compared to other providers, and verify the future creation dates align with actual model release schedules

greptile-apps

_{8 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2025-11-25T01:25:26Z

packages/cost/models/authors/openai/gpt-5.1/models.ts

    tokenizer: "GPT",
  },
+  "gpt-5.1-2025-11-13": {
+    name: "OpenAI GPT-5.1",


style: Model name is identical to the base model (line 5). Consider adding version info to distinguish this specific dated version, e.g., 'OpenAI GPT-5.1 (2025-11-13)'.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

Prompt To Fix With AI

This is a comment left during a code review. Path: packages/cost/models/authors/openai/gpt-5.1/models.ts Line: 57:57 Comment: **style:** Model name is identical to the base model (line 5). Consider adding version info to distinguish this specific dated version, e.g., 'OpenAI GPT-5.1 (2025-11-13)'. <sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub> How can I resolve this? If you propose a fix, please make it concise.

added more model versions

03129b5

vercel bot deployed to Preview – helicone-eu November 25, 2025 01:22 View deployment

greptile-apps bot reviewed Nov 25, 2025

View reviewed changes

vercel bot deployed to Preview – helicone-bifrost November 25, 2025 01:25 View deployment

vercel bot deployed to Preview – helicone November 25, 2025 01:28 View deployment

H2Shami approved these changes Nov 25, 2025

View reviewed changes

fixing shit

6bc03c4

vercel bot deployed to Preview – helicone-eu November 25, 2025 05:33 View deployment

vercel bot deployed to Preview – helicone-bifrost November 25, 2025 05:36 View deployment

vercel bot deployed to Preview – helicone November 25, 2025 05:38 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

added more model versions #5293

added more model versions #5293

Uh oh!

chitalian commented Nov 25, 2025

Uh oh!

vercel bot commented Nov 25, 2025 •

edited

Loading

Uh oh!

claude bot commented Nov 25, 2025 •

edited

Loading

Uh oh!

greptile-apps bot commented Nov 25, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

greptile-apps bot Nov 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

added more model versions #5293

Are you sure you want to change the base?

added more model versions #5293

Uh oh!

Conversation

chitalian commented Nov 25, 2025

Ticket

Component/Service

Type of Change

Testing

Technical Considerations

Dependencies

Deployment Notes

Context

Screenshots / Demos

Misc. Review Notes

Uh oh!

vercel bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

claude bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Complete

Review Analysis

Critical Issues Found

Code Quality Issues

Security & Performance

Positive Aspects

Uh oh!

greptile-apps bot commented Nov 25, 2025

Greptile Overview

Greptile Summary

Important Files Changed

Confidence score: 4/5

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Nov 25, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vercel bot commented Nov 25, 2025 •

edited

Loading

claude bot commented Nov 25, 2025 •

edited

Loading