Skip to content

feat: add manual thinking budget control for supported providers/models #3892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Ruakij
Copy link

@Ruakij Ruakij commented May 23, 2025

Related GitHub Issue

Closes: #3008

Description

This PR introduces manual control for the "thinking token limit" if supported. Not setting it will use default handling by provider/model e.g. Gemini will then adjust the budget according to the request as their docs.
Support for this feature is determined by checking if modelInfo has modelMaxThinkingTokens. (from a quick glance, this seems to correlate with the only providers for which we support it currently: gemini and vertex)

Key changes:

  • Added manualThinkingBudgetEnabled setting (defaults to false), which sets max thinking token limit to undefined when false, resulting it it not being added to the request.
  • Migrate settings for existing profiles with modelMaxThinkingTokens already set (sets manualThinkingBudgetEnabled to true).
  • Add Checkbox to enable manual control before the thinking tokens slider.
  • Add Tests for UI

Test Procedure

  1. Unit-Tests
  2. Manual:
    • In provider settings, select a model with thinking support.
    • Default (Automatic Budget): "Manually control" checkbox is unchecked. Slider is hidden.
    • Manual Override: Check "Manually control". Slider appears. Set budget.
    • Unsupported Providers/Models (e.g. OpenRouter with sonnet 3.7 thinking): Checkbox is disabled.
    • Migration: Existing configs with modelMaxThinkingTokens should have manual control enabled.

Type of Change

  • 🐛 Bug Fix: Non-breaking change that fixes an issue.
  • New Feature: Non-breaking change that adds functionality.
  • 💥 Breaking Change: Fix or feature that would cause existing functionality to not work as expected.
  • ♻️ Refactor: Code change that neither fixes a bug nor adds a feature.
  • 💅 Style: Changes that do not affect the meaning of the code (white-space, formatting, etc.).
  • 📚 Documentation: Updates to documentation files.
  • ⚙️ Build/CI: Changes to the build process or CI configuration.
  • 🧹 Chore: Other changes that don't modify src or test files.

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • [ x Self-Review: I have performed a thorough self-review of my code.
  • Code Quality:
    • My code adheres to the project's style guidelines.
    • There are no new linting errors or warnings (npm run lint).
    • All debug code (e.g., console.log) has been removed.
  • Testing:
    • New and/or updated tests have been added to cover my changes.
    • All tests pass locally (npm test).
    • The application builds successfully with my changes.
  • Branch Hygiene: My branch is up-to-date (rebased) with the main branch.
  • Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
  • Changeset: A changeset has been created using npm run changeset if this PR includes user-facing changes or dependency updates.
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

Unchecked:
Screenshot_20250523_104902
Checked:
Screenshot_20250523_104914
Disabled on OpenRouter:
Screenshot_20250523_105128

Documentation Updates

  • Yes, documentation updates are required.
    • Explain new "Manually control thinking token limit" setting, default automatic behavior, and manual override. (if exists)

Additional Notes

Get in Touch

Discord: ruakij

@Ruakij
Copy link
Author

Ruakij commented May 23, 2025

-.- Rework of reasoning was just merged, guess i need to adjust this for the rework

@hannesrudolph hannesrudolph moved this from New to PR [Draft/WIP] in Roo Code Roadmap May 23, 2025
@hannesrudolph
Copy link
Collaborator

hannesrudolph commented May 23, 2025

@Ruakij sorry about that!

@hannesrudolph hannesrudolph moved this from PR [Draft / In Progress] to TEMP in Roo Code Roadmap May 26, 2025
@daniel-lxs daniel-lxs moved this from TEMP to PR [Draft / In Progress] in Roo Code Roadmap May 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: PR [Draft / In Progress]
Development

Successfully merging this pull request may close these issues.

Support gemini-2.5-flash determining its own thinking budget
3 participants