Skip to content

Add metadata to LLM requests #3830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
4 tasks done
dtrugman opened this issue May 22, 2025 · 2 comments · Fixed by #3832
Closed
4 tasks done

Add metadata to LLM requests #3830

dtrugman opened this issue May 22, 2025 · 2 comments · Fixed by #3832
Labels
enhancement New feature or request Issue - In Progress Someone is actively working on this. Should link to a PR soon.

Comments

@dtrugman
Copy link

What problem does this proposed feature solve?

This change would enable the model routers to provide better and more detailed dashboards for Roo Code users, including:

  • Spending statistics on task level
  • Spending statistics on mode level
  • Statistics about min, avg and max turns per task
  • Visibility around compression

Describe the proposed solution in detail

Add a new metadata argument to the createMessage method of the ApiHandler interface.
The metadata would allow for an optional Metadata type that contains the taskId and the current mode.

Different providers can then either forward this information when making the LLM chat completions call, or do nothing with it.

Technical considerations or implementation details (optional)

This should have 0 implications on the actual performance of the platform, as the data is already accessible from Task.ts, where most of the calls are made.

Describe alternatives considered (if any)

Idea no.1:
Try and get this information from within the relevant provider.
Why not:
Inverses the responsibilities and creates an unhealthy coupling.

Idea no.2:
Pass the information as a Record<string,any>.
Why not:
Passing discrete and type-checked values makes more sense while there's no fragmentation.

Additional Context & Mockups

No response

Proposal Checklist

  • I have searched existing Issues and Discussions to ensure this proposal is not a duplicate.
  • This proposal is for a specific, actionable change intended for implementation (not a general idea).
  • I understand that this proposal requires review and approval before any development work begins.

Are you interested in implementing this feature if approved?

  • Yes, I would like to contribute to implementing this feature.
@dtrugman dtrugman added the enhancement New feature or request label May 22, 2025
@hannesrudolph hannesrudolph moved this from New to Issue [In Progress] in Roo Code Roadmap May 23, 2025
@hannesrudolph hannesrudolph added the Issue - In Progress Someone is actively working on this. Should link to a PR soon. label May 23, 2025
@dtrugman
Copy link
Author

@hannesrudolph, any idea who should we involve here to get this moving?

@mrubens
Copy link
Collaborator

mrubens commented May 25, 2025

Makes sense to me

@github-project-automation github-project-automation bot moved this from Issue [In Progress] to Done in Roo Code Roadmap May 25, 2025
@github-project-automation github-project-automation bot moved this from New to Done in Roo Code Roadmap May 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Issue - In Progress Someone is actively working on this. Should link to a PR soon.
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants