- 
                Notifications
    You must be signed in to change notification settings 
- Fork 2.8k
feat: #1994 Add a list of per-request usage data to Usage #1995
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: #1994 Add a list of per-request usage data to Usage #1995
Conversation
| @habema 👍 for this small useful feature. I often need the token usage of the last request to estimate how much of the context window is being consumed, rather than looking at the total sum. By the way, the naming in your PR description doesn’t match the code: 
 Personally, I feel  | 
| @ihower thank you. Those mismatches are because of the code review, directly above your comment. | 
| @rm-openai Let me know if you have any comments. I will merge this within a few business days. | 
| @ihower Thanks for sharing your thoughts. 
 request_usages was actually one of the name candidates on my end, but ChatGPT pointed out that it might sound a bit unnatural in English LOL. So, I leaned toward using request_usage_xxx (like entries, list, etc.) instead. | 
Resolves #1994
Changes:
RequestUsageclass to store per-request detailsrequest_usage_entriesfield toUsageclassUsage.add()to automatically create per-request entries for single requestsSample
Usageobject after changes: