Show token usage for a conversation in the chat UI

[garland3]

Summary

There is currently no way for a user to see how many tokens a conversation has consumed. Users hit context limits or incur unexpected cost with no warning and no visibility into which turns or tools are expensive.

Add token usage visibility to the chat UI, backed by the usage data already returned by the model provider on each response.

Motivation

• Users cannot tell how close a conversation is to the model's context window.
• Long tool outputs (MCP results, file reads, RAG chunks) can silently dominate the context; today there is no signal until something breaks.
• Cost awareness: per-conversation usage is the minimum unit people reason about.

Proposed behavior

1. Per-message usage — each assistant message can expose its input / output / (and cached, if reported) token counts, shown on demand rather than always-on clutter.
2. Conversation total — a running total for the conversation, visible in the chat header or side panel.
3. Context window indicator — total prompt tokens relative to the selected model's context limit (e.g. 18.2k / 200k), so users can see when they are approaching the limit.
4. Model switching — the indicator updates when the selected model changes, since the limit differs per model.

Acceptance criteria

• The backend captures token usage (input, output, and cached/reasoning counts when the provider reports them) from each model response and includes it in the message payload sent to the frontend.
• Usage is persisted with the conversation so totals survive a page reload / conversation reopen.
• The frontend displays a per-message token count for assistant messages.
• The frontend displays a running conversation total (input + output) that updates as new messages stream in.
• A context-window indicator shows current prompt tokens vs. the selected model's limit, and reflects the currently selected model.
• Usage display degrades gracefully when a provider does not report usage (no crash, no misleading zeros — show "unavailable" or omit).
• The feature is covered by backend tests for usage extraction/persistence and frontend tests for rendering totals.
• Documentation updated to describe where token usage appears.

Out of scope

• Per-user or org-wide usage dashboards / quotas and billing enforcement.
• Automatic context compaction or trimming (worth a separate issue).

[garland3]

For the context-window indicator, where should the per-model limit come from? llmconfig.yml has no context_window field today — the limits only appear as prose inside model_card. Should an explicit per-model key be added and backfilled, or is it expected to come from somewhere else?

---

Posted automatically by AGENT-REVIEW-BOT-3