Skip to content

feat: add token observability with per-entity lifecycle attribution#194

Open
ChenNima wants to merge 2 commits into
developfrom
feat/token-observability
Open

feat: add token observability with per-entity lifecycle attribution#194
ChenNima wants to merge 2 commits into
developfrom
feat/token-observability

Conversation

@ChenNima
Copy link
Copy Markdown
Contributor

Summary

  • Track AI agent token consumption across the full AI-DLC lifecycle (elaboration → proposal → review → execution → verify)
  • Shell hooks extract per-API-call usage from CC transcripts with streaming dedup and delta cache_read
  • Sub-agents: last-turn snapshot matching CC's total_tokens; main agent: per-turn delta with user-boundary round filtering
  • Primary entity model for sub-agents (task > proposal > idea > document priority)
  • isReviewer field distinguishes task-reviewer (verify) from dev (execution)
  • Frontend tokensSum includes all 4 Claude API fields (input + output + cache_create + cache_read)

Changed files

Area Files Change
Schema prisma/schema.prisma, 3 migrations Add ToolUsageEvent, TokenUsageRecord models, isReviewer field
Shell hooks on-stop.sh, on-subagent-stop.sh, on-session-end.sh, on-post-tool-log.sh Token + tool usage extraction and upload
API routes agent-report/token-usage, agent-report/tool-usage, observability/* Token ingestion + query endpoints
Service observability.service.ts Attribution engine, phase bucketing, lifecycle queries
Frontend tokens-view.tsx, agent-observability.tsx, token-usage-card.tsx, task-tokens-view.tsx Observability UI components
i18n messages/en.json, messages/zh.json Observability labels
Tests token-attribution.test.ts, observability.service.test.ts, format-tokens.test.ts 16 + 23 + 25 = 64 new tests

Test plan

  • npx tsc --noEmit passes
  • pnpm test — 1363 tests pass
  • bash public/chorus-plugin/bin/test-syntax.sh — 16/16 pass
  • E2E verified: proposal reviewer tokens match CC (22k), task reviewer match (28.8k), main agent phases (elaboration/proposal/execution) all non-zero and correctly attributed
  • Non-chorus conversation turns filtered out (no leakage to execution phase)

🤖 Generated with Claude Code

ChenNima and others added 2 commits April 20, 2026 13:28
…le tracking

Track AI agent token consumption across the full AI-DLC lifecycle (elaboration → proposal → review → execution → verify). Key design:

- Shell hooks extract per-API-call usage from CC transcripts with streaming dedup and delta cache_read computation
- Sub-agents use last-turn snapshot (matches CC's total_tokens); main agent uses per-turn delta with user-boundary round filtering to prevent non-chorus conversation leakage
- Primary entity model for sub-agents (task > proposal > idea > document priority)
- Carry-forward attribution for main agent with user message boundaries as reset points
- isReviewer flag distinguishes task-reviewer (verify phase) from dev (execution phase)
- Delete-then-insert upsert on sourceSessionId for idempotent uploads
- Frontend tokensSum includes all 4 Claude API fields (input + output + cache_create + cache_read)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sage

Brings observability.service.ts statement coverage above the 95% CI threshold.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 96.62% (🎯 95%) 2318 / 2399
🔵 Statements 95.59% (🎯 95%) 2515 / 2631
🔵 Functions 94.88% (🎯 93%) 464 / 489
🔵 Branches 87.17% (🎯 85%) 1536 / 1762
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/lib/format-tokens.ts 100% 100% 100% 100%
src/services/observability.service.ts 96.28% 83.15% 100% 96.66% 131, 376, 460-463, 467, 511-514
Generated in workflow #431 for commit ef96e92 by the Vitest Coverage Report Action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant