Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 2, 2025

When using tools with track_usage=True, only the final API call's usage was tracked, underreporting token consumption and costs across multi-turn tool loops.

Changes

  • Modified _run_tool_loop: Returns all API responses from every iteration, not just the final response
  • Added _accumulate_usage_metrics: Sums token counts (input, output, cached) across all API responses in tool loops
  • Updated query_unified: Uses accumulated metrics when tools are present, single-call metrics otherwise

Example

# Previously: only final API call tracked
result = agent.query_unified(
    message="Get data and analyze it",
    tools=[get_data_tool, analyze_tool],
    track_usage=True
)
# result.usage.input_tokens would undercount by missing tool iteration tokens

# Now: cumulative tracking across all calls
result = agent.query_unified(
    message="Get data and analyze it", 
    tools=[get_data_tool, analyze_tool],
    track_usage=True
)
# result.usage.input_tokens = sum of all iterations (e.g., 100 + 150 = 250)
# result.usage.output_tokens = sum of all iterations (e.g., 20 + 30 = 50)

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI self-assigned this Dec 2, 2025
Copilot AI mentioned this pull request Dec 2, 2025
@dosumis dosumis marked this pull request as ready for review December 2, 2025 14:59
Copy link
Contributor Author

Copilot AI commented Dec 2, 2025

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • astral.sh
    • Triggering command: /usr/bin/curl curl -LsSf REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] WIP address feedback from review on unit test fixes Accumulate usage metrics across tool call iterations Dec 2, 2025
Copilot AI requested a review from dosumis December 2, 2025 15:09
@dosumis dosumis merged commit 2929e94 into unify_queries Dec 2, 2025
@dosumis dosumis deleted the copilot/sub-pr-11 branch December 2, 2025 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants