📊 pi-tps

Tokens-per-second tracker for pi

Generation speed, TTFT, stall detection, and cost — after every agent turn.

---

Originally from badlogic/pi-mono. Packaged as an installable pi extension.

---

Quick start

pi install npm:@monotykamary/pi-tps

What's included

Extension	Tracks TPS, TTFT, stall time, token usage, and cost after each turn
Export	/tps-export command — dump telemetry as JSONL with session structure

Features

• Accurate TPS: Uses performance.now() sub-millisecond timing; excludes TTFT, tool-execution gaps, and network latency from generation speed
• Stall detection: Detects inference pauses (GPU queuing, request queuing) and subtracts them from generation TPS — no inflated rates
• Burst discrimination: Distinguishes genuine streaming from buffer-flush dispatch; shows — when the rate is structurally unidentifiable
• Multi-message turns: Aggregates tokens and timing across tool-call chains within one turn
• Notification banner: Shows a transient popup with TPS, TTFT, total time, tokens, and stalls
• Persisted notifications: Restored on session resume and /tree navigation (structured + legacy backward compatible)
• Export command: Dump telemetry as JSONL with automatic tree re-chaining for web inspectors
• Extensible: Emits tps:telemetry events so other extensions can react to telemetry

Install

pi install npm:@monotykamary/pi-tps

Or install from GitHub:

pi install https://github.com/monotykamary/pi-tps

The published npm package is intentionally slim: only extensions/pi-tps/index.ts, README.md, and LICENSE. It has no runtime dependencies and does not ship npm-shrinkwrap.json / lockfiles, so installs do not pull the multi-hundred-MB developer toolchain into ~/.pi/agent/npm.

Manual install

cp -r extensions/pi-tps ~/.pi/agent/extensions/

Then /reload in pi.

---

Output format

TPS 42.5 tok/s · TTFT 1.2s · 29.7s · in 567 · out 1.2K · stall 4.3s×1 · $3.60/M

Field	Description
TPS	Tokens per second (generation speed, excludes TTFT & stalls)
TTFT	Time to first token (seconds, 1 decimal)
s	Total wall-clock time from request to completion
in	Input tokens (human-readable: K/M/B)
out	Output tokens (human-readable: K/M/B)
stall	Accumulated stall time × stall count (shown only when stalls exist)
$/M	Blended $/million-tokens rate for this turn (shown only when a cost source is available)

When TPS can't be determined (burst delivery, too few chunks), the field shows —:

TPS — · TTFT 0.8s · 1.3s · in 291 · out 46 · $2.59/M

$/M-tokens rate

The trailing $/M segment is a blended per-turn rate: effectiveCost / (totalTokens / 1_000_000), shown to two decimals. Only one cost source contributes per turn (no double-counting):

• When pi-neuralwatt-provider reports the energy-billed cost for the turn (what you're actually invoiced), that wins outright.
• Otherwise the list-price compute cost from message.usage.cost.total (derived from the model's pricing table via pi-ai) is used.
• When neither is available (no pricing, zero tokens), the segment is omitted.

The cross-extension handoff is event-based: pi-neuralwatt-provider emits neuralwatt:turn-energy with { costUsd, energyJoules, turnIndex } in its own turn_end; pi-tps subscribes at load and stashes costUsd keyed by turnIndex. pi dispatches turn_end handlers sequentially, so if the neuralwatt provider was registered before pi-tps the value is already cached when pi-tps's turn_end runs; if not, pi-tps falls back to the list-price rate for that one turn (the neuralwatt widget still shows billing separately). No latency is added — pi-tps reads the cache synchronously, never awaits.

Human-readable scaling (for token counts):

• < 1K: raw integer (567)
• ≥ 1K: one decimal, drops .0 (1.2K, 2K, 15.3K)
• ≥ 1M: same pattern (1.5M)
• ≥ 1B: same pattern (1.2B)

Duration formatting:

• < 60s: one decimal (2.3s, 45.0s)
• ≥ 60s: up to two units with no decimals (1m 30s, 2h 15m, 3d 12h, 1w 3d, 1mo 0d, 1y 0d)

---

How it works

The extension hooks into pi's lifecycle events. The critical detail: message_start fires at stream creation (before any tokens), so TTFT is measured at the first message_update, which carries the first real token content.

Event sequence

turn_start         →  request sent to LLM, timer starts
message_start      →  stream created, stall-tracking reset for this message
message_update (1) →  first token arrives → TTFT captured
message_update (N) →  streaming tokens arrive → inter-update span & stall detection
message_end        →  message complete, generation time accumulated
turn_end           →  telemetry computed and displayed

Timing breakdown

Phase	Measured by
TTFT	turn_start → first message_update
Generation	per-message wall clock (message_start → message_end), summed across messages in the turn
Stream span	first message_update (post-TTFT) → last message_update — the pure streaming window
Total	turn_start → last message_end in the turn

This approach excludes:

• Network latency (included in TTFT)
• Tool-execution gaps between messages (stall clock resets on each message_start)
• Server queue time (included in TTFT)

Stall detection

Every message_update (after TTFT) measures the gap since the last update. Gaps ≥ 500ms are classified as inference stalls:

• The full gap is accumulated as stallMs
• Consecutive stalled updates count as one stall event
• Stalls are subtracted from the streaming window when computing generation TPS
• The stall clock resets at each message_start, so tool-execution gaps between messages are never counted as stalls

When a stall occurs before the first stream update (common in request-queuing scenarios), the TPS algorithm detects the artifact and falls back to a conservative estimate rather than producing an inflated rate.

TPS algorithm (three-branch gate)

The extension uses a defense-in-depth strategy to produce reliable TPS:

1. Primary — Requires ≥5 streaming updates with ≥1ms average inter-chunk gap and stall time < active generation time. Subtracts stalls from the streaming window for pure generation speed.

2. Fallback — When primary conditions fail but ≥2 updates exist and total generation time ≥50ms. Uses the full generation window (includes TTFT, so it underestimates — safe by design). Applies partial stall reduction when stalls dominate.

3. Null — Returns null (displayed as —) when the timing is structurally unidentifiable: burst delivery (all tokens arrive in the same tick), too few chunks, or generation time too short for a reliable rate.

---

Rehydration

When you resume a session (or navigate branches with /tree), pi-tps restores the most recent TPS notification — so you can see your last turn's stats after a reload.

Supports both the current structured TurnTelemetry format and legacy { message, timestamp } entries for backward compatibility with session files created by earlier versions.

---

Export command

Dump telemetry as JSONL for inspection or analysis:

/tps-export             # current branch, all custom entries
/tps-export --full      # all branches in the session
/tps-export tps         # current branch, filter by customType "tps"
/tps-export tps --full  # all branches, filter by customType "tps"

Each exported file is written to ~/.cache/pi-telemetry/pi-telemetry-{scope}-{sessionId}-{timestamp}.jsonl.

The exporter includes structural entries (model_change, branch_summary) alongside telemetry entries so the exported tree is fully resolvable — the web inspector can show model switches and branch points. Parent IDs are automatically re-chained to point to the nearest ancestor that's included in the export, producing a self-contained tree.

---

Telemetry event

After each turn, pi-tps emits a tps:telemetry event on pi's shared event bus. Other extensions can listen to build custom widgets, dashboards, or cost trackers.

pi.events.on('tps:telemetry', (data) => {
  // data matches the TurnTelemetry structure below
  console.log(data.tps, data.tokens, data.timing);
});

The event payload:

Field	Type	Description
tps	number | null	Tokens per second, or null when unidentifiable
model.provider	string	Provider name (e.g. openai)
model.modelId	string	Model identifier (e.g. gpt-4)
tokens.input	number	Input tokens (summed across all assistant messages)
tokens.output	number	Output tokens generated by the LLM
tokens.cacheRead	number	Cache-read tokens (provider-dependent)
tokens.cacheWrite	number	Cache-write tokens (provider-dependent)
tokens.total	number	Total tokens (input + output + cache)
timing.ttftMs	number | null	Time to first token in milliseconds
timing.totalMs	number	Total wall-clock time from request to completion
timing.generationMs	number	Streaming wall clock (message_start → message_end)
timing.streamMs	number | null	Inter-update span: first → last streaming update
timing.stallMs	number	Accumulated inference stall time in ms
timing.stallCount	number	Number of discrete stall events
timing.messageCount	number	Assistant messages in this turn
cost.input	number | null	Input token cost
cost.output	number | null	Output token cost
cost.cacheRead	number | null	Cache-read token cost
cost.cacheWrite	number | null	Cache-write token cost
cost.total	number | null	Total cost for this turn
timestamp	number	Unix timestamp (ms) when telemetry was computed
rateUsdPerMTokens	number | null	Blended $/M-tokens rate for this turn (Neuralwatt billed cost when available, else list-price compute cost). null when no cost source is usable or tokens are zero.

When cost is unavailable (provider doesn't report it), the entire cost object is null.

---

Testing

Dev tooling (vitest, typescript, hooks) stays in the git repo only — use pnpm for local development.

# Install dev dependencies (repo checkout only)
pnpm install

# Run tests
pnpm test

# Run tests with coverage
pnpm test:coverage

# Type check
pnpm typecheck

# Verify published tarball contents (should be ~3 runtime files)
pnpm pack:check

---

License

MIT