Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Awesome LLM Token Optimization Awesome

A curated list of strategies, tools, papers, and resources for reducing LLM token costs and improving efficiency in production.

Building with LLMs is expensive. An agent processing 10 reasoning steps can consume 50K-100K tokens per task. This list collects everything you need to cut costs by 80-99% without sacrificing quality.

Contents


Quick Wins

The highest-impact strategies ranked by effort-to-savings ratio:

Strategy Savings Effort Link
Prompt caching 90% input tokens Add cache headers Prompt Caching
Token-efficient tool use 70% output reduction Flip a flag Prompt Engineering for Efficiency
Batch API 50% Queue non-urgent work Batch APIs
Model routing 60-95% Route by task complexity Model Routing
Response caching 100% on repeats Add a cache layer Comprehensive Guides
Prompt compression 5-20x Use LLMLingua Prompt Compression

Combined pipeline: Cache prefix (90%) + route to cheapest model (60-95%) + batch non-urgent (50%) + compress prompts (5-20x) + cache responses (100% on repeats) = 95-99% cost reduction vs. naive approach.

Prompt Caching

Reuse previously-processed prompt prefixes to avoid re-computing the same tokens.

Provider Docs

Strategy: Cached Prefix Pattern

Structure prompts so the system prompt + user profile is the first ~2,000 tokens. All subsequent calls share this prefix. For bulk operations (e.g., scoring 50 items): 1x full + 49x at 10% = 88% total savings.

Tools

  • autocache - Transparent Anthropic proxy that auto-injects cache_control breakpoints at optimal positions; up to 90% cost and 85% latency reduction. Stars

Batch APIs

50% discounts for non-time-critical requests. Combine with caching for 95% savings.

Model Routing

Route simple tasks to cheaper models. 80% of typical LLM calls don't need the most expensive model.

Frameworks

  • RouteLLM - Open-source LLM router by LMSYS. Trains routers from preference data; 2x+ cost reduction. Note: last commit Aug 2024; LMSYS team shifted focus to Chatbot Arena. Stars
  • LiteLLM - SDK + proxy for 100+ LLMs with routing, cost tracking. Strategies: least-busy, cost-based, latency-based. Stars
  • NotDiamond - Per-query best-model selection. Note: the Python SDK was archived Dec 2025 (read-only); the notdiamond.ai service remains active. Stars
  • Bifrost - 50x faster than LiteLLM; adaptive load balancer, 1000+ models. Stars
  • vLLM Semantic Router - System-level signal-driven router for Mixture-of-Models across cloud, data center, and edge. v0.3 "Themis" (June 2026); SAAR adds session-aware model selection cutting model switches 79% in multi-agent deployments. Stars
  • LLMRouter - Open-source routing library with 16+ routers (single-round, multi-turn, agentic, personalized) and a unified CLI. Stars
  • Portkey AI Gateway - Open-source AI gateway routing to 1,600+ LLMs with guardrails, caching, and load balancing. Acquired by Palo Alto Networks (May 2026); gateway remains open-source under Apache 2.0. Stars
  • OpenRouter - Unified API for 300+ models with auto-router.
  • Martian Router - Patent-pending; cuts costs 20-97% via "Model Mapping".

Curated Lists

Research

Prompt Compression

Reduce prompt size while preserving information quality.

Tools

  • LLMLingua - Up to 20x compression. Coarse-to-fine iterative method. Integrates with LangChain/LlamaIndex. Stars
  • Headroom - Compress tool outputs, logs, files, and RAG chunks before they reach the LLM (60-95% fewer tokens); library, proxy, and MCP server. Claude Code/Cursor/Aider compatible.
  • code2prompt - Codebase to LLM prompt with token counting. Stars
  • RTK - Single-binary Rust CLI proxy that compresses dev-command output 60-90% before it reaches a coding agent's context. Works with Claude Code, Cursor, Copilot, Gemini CLI. Stars
  • TOON - Token-Oriented Object Notation: a compact, schema-aware encoding for passing JSON-like data to LLMs; 30-60% fewer tokens than JSON on uniform arrays of objects. Stars
  • llmtrim - Quality-gated local proxy and MCP server that compresses prompts, tool outputs, and replies before they reach the LLM, reverting any step that doesn't save tokens (project-reported -31% input / -74% output across 112 A/B cases). Rust CLI plus multi-language library bindings and a WebAssembly/JS package. Stars
  • lean-ctx - Rust binary context intelligence layer for AI coding agents; 60-90% fewer tokens via shell-output compression and 10 cached-read modes; MCP server with 76 tools and cross-session memory. Works with Claude Code, Cursor, Copilot, Windsurf, Gemini CLI, and 30+ others. Stars

Research

The full prompt-compression paper table lives in the Academic Papers section below.

Guides

Lossless Compression Principles

Rule-based lossless distillation achieves 3-4:1 compression without any model:

Strip prose transitions, hedging, rhetoric, and common knowledge. Preserve numbers, entities, decisions, constraints, and risks. Transform prose into dense bullets and verbose text into semicolon-joined clauses. Split into 3,000-5,000 token self-contained sections that load independently.

Context Window Management

Key Research

Provider Docs

Chunking & Splitting

KV Cache Optimization

Server-side optimizations for inference efficiency.

Inference Engines

  • vLLM - PagedAttention, high-throughput inference. Stars
  • SGLang - RadixAttention for automatic KV cache reuse. Stars
  • GPUStack - GPU cluster manager for vLLM/SGLang. Stars
  • NVIDIA Dynamo - Datacenter-scale distributed inference with KV-cache-aware routing and disaggregated prefill/decode; ~2x faster TTFT, 7x throughput/GPU. Stars
  • llm-d - Kubernetes-native distributed serving with prefix-cache-aware routing and tiered KV offload to CPU/disk (3x output throughput). Stars
  • Mooncake - Distributed KVCache engine (the serving platform behind Moonshot AI's Kimi); integrated into vLLM for high-bandwidth KV-cache transfer and cross-instance prefix reuse across disaggregated prefill/decode. Stars

Compression Tools

  • NVIDIA kvpress - KV cache compression made easy. Stars
  • R-KV - Redundancy-aware compression (NeurIPS 2025). Stars
  • llm-compressor - Compression for deployment with vLLM. Stars
  • NVIDIA Model Optimizer - Quantization, pruning, distillation, speculative decoding. Stars
  • TurboQuant - Google's ICLR 2026; 5x KV cache compression.
  • aibrix - Cost-efficient infrastructure for GenAI inference. Stars
  • LMCache - KV cache layer for vLLM/SGLang; offloads and reuses caches across engines (CPU/disk/S3) to cut TTFT for long-context, multi-turn, and RAG. Stars
  • kvcached - Virtualized elastic KV cache decoupling virtual/physical GPU memory for dynamic GPU sharing; plugs into vLLM/SGLang. Stars
  • KVzip - Query-agnostic KV cache eviction via context reconstruction (NeurIPS 2025 Oral); 3-4x memory reduction, 2x lower latency. Stars
  • DeepSpec - DeepSeek's open-source speculative decoding stack powering DSpark; 60-85% faster on V4-Flash, 57-78% on V4-Pro; outperforms Eagle-3. MIT license. Stars

Research

The full KV-cache paper table lives in the Academic Papers section below.

Educational

Browser Tool Efficiency

Different browser automation approaches consume vastly different context.

Agent Output Size Efficiency Link
WebFetch ~1.5 KB (AI-summarized) 20x better Docs
Playwright MCP ~10-33 KB (accessibility tree) Baseline GitHub
Agent Browser ⚠️ ~28 KB (accessibility tree) Project unmaintained 2026-05 — superseded by browser-use direct mode + Playwright MCP GitHub
Lightpanda ~16 KB (raw markdown) 2x better GitHub

For 10-page workflows: WebFetch = ~15KB vs Playwright = ~330KB total context consumed.

Why Accessibility Trees Are Efficient

The accessibility tree strips visual styling to retain only semantic structure (name, role, state, value). 10-50x smaller than raw HTML. See: Token cost analysis in browser MCPs.

Further Reading

Cost Tracking Tools

  • Langfuse - Open-source LLM observability + cost tracking. Cost tracking docs. Acquired by ClickHouse (Jan 2026); still actively developed, MIT-licensed. Stars
  • Helicone - LLM observability, 300+ models, SOC 2. Cost tracking cookbook. Acquired by Mintlify (Mar 2026); now maintenance-only — security/bug fixes and new-model support continue, no new feature work. Stars
  • LiteLLM Spend Tracking - Per-key/team spend tracking and budget routing for the LiteLLM proxy across 100+ LLMs. Stars
  • tokencost - USD cost estimates for 400+ LLMs. Stars
  • AgentOps - Agent monitoring with LLM cost tracking. Stars
  • agenttrace - Local-first TUI that reads Claude Code / Codex / Gemini / Aider / Cursor sessions to surface tokens, cost, cache use, retries, and latency. Stars
  • Future AGI traceAI - OpenTelemetry-based AI tracing capturing per-call tokens, cost, and latency across 35+ frameworks. Stars
  • ccusage - Fast local CLI reporting tokens and cost across 14+ coding agents (Claude Code, Codex, Gemini CLI, Copilot); offline, no upload. Stars
  • OpenLLMetry - OpenTelemetry-based GenAI observability instrumenting LLM and vector-DB calls with per-call token and latency telemetry. Stars
  • MLflow - Open-source AI/ML platform with GenAI observability in MLflow 3.x: LLM call tracing with per-span token tracking, prompt optimization tooling, and AI Gateway integration for cost control. Stars
  • Helicone AI Gateway - Fastest open-source AI gateway (Rust). Stars
  • Anthropic Token Counter - Free pre-flight token counting endpoint.
  • tiktoken - OpenAI's fast BPE tokenizer (Python/Rust), 3-6x faster.
  • LangSmith Cost Tracking - Automatic recording with dashboards.
  • LlamaIndex Cost Analysis - Estimate costs before calls.

Pricing Comparison

Live Pricing Tools

Provider Pricing Pages

Notable Recent Pricing (June–July 2026)

Model Input /MTok Output /MTok Notes
Claude Fable 5 $10.00 $50.00 Anthropic's most capable model; 1M context (June 2026). Access suspended June 12 via US export-control directive; restored July 1, 2026.
Claude Opus 4.8 $5.00 $25.00 1M context at standard pricing
Claude Sonnet 5 $2.00 $10.00 Introductory pricing through Aug 31, 2026 (standard: $3/$15 per MTok); 1M context; most agentic Sonnet; launched June 30, 2026.
GPT-5.5 $5.00 $30.00 OpenAI flagship; 1M context; 90% cached-input discount
GPT-5.4 $2.50 $15.00 Half the cost of GPT-5.5; 50% Batch API discount
DeepSeek V4 Flash $0.14 $0.28 Cheapest frontier; 98% cache savings
DeepSeek V4 Pro $0.435 $0.87 1M context; thinking + non-thinking modes
Gemini 3.1 Pro $2.00 $12.00 Preview since Feb 2026; ≤200K context; doubles to $4/$18 above 200K tokens
Gemini 3.5 Flash $1.50 $9.00 Launched May 19, 2026; 1M context window
Gemini 2.5 Flash-Lite $0.10 $0.40 Budget option

Tokenizer note (Anthropic): Claude Opus 4.7+, Sonnet 5, and Fable 5 use a newer tokenizer that produces roughly 30% more tokens for the same text; per-token prices are unchanged, so the effective cost of a fixed input rises proportionally (Sonnet 4.6 and earlier keep the previous tokenizer). Benchmark your real workload before assuming a newer model lowers cost — confirmed on Anthropic's official pricing docs (already linked under Provider Pricing Pages above).

Prompt Engineering for Efficiency

Official Guides

Community

Concise Reasoning Research

Comprehensive Guides

Academic Papers

Prompt Compression

Paper Year Key Result
Prompt Compression Survey 2024 Comprehensive survey of all techniques
LLMLingua 2023 Up to 20x compression (EMNLP)
LLMLingua-2 2024 3-6x faster via BERT distillation (ACL)
LongLLMLingua 2023 4x fewer tokens in long contexts
Selective Context 2023 50% reduction via self-information pruning
RECOMP 2023 5% token ratio for retrieved docs
500xCompressor 2024 6-480x compression ratios
LoPace 2026 Lossless; 72.2% savings
SCOPE 2025 Training-free generative rewriting
Dynamic Compressing 2025 MDP-based adaptive token removal
Empirical Study 2025 Benchmarks 6 methods across 13 datasets
Telegraph English 2026 Symbolic rewriting protocol; ~50% token reduction at 99.1% accuracy; outperforms LLMLingua-2 at matched ratio
Prompt Compression in the Wild 2026 First large-scale production study (30K queries) of the latency vs. quality tradeoff
Production Compression RCT 2026 Pre-registered randomized trial: moderate compression −27.9% cost; over-compression backfires
LongCodeZip 2025 Code-aware two-stage compression; up to 5.6x with no performance loss (ASE 2025)
Behavior-Equivalent Token 2025 Distills a long system prompt into one learned token; no aux model or labels
SAC (Semantic Anchors) 2025 Autoencoding-free context compression via selected anchor tokens; no compression-token pretraining

Model Routing & Cascading

Paper Year Key Result
FrugalGPT 2023 Seminal cascade paper; up to 98% cost reduction
RouteLLM 2024 2x+ cost reduction without quality loss
Hybrid LLM 2024 40% fewer calls to large model
Unified Routing + Cascading 2024 +14% over individual strategies
Dynamic Routing Survey 2026 Comprehensive survey
Pay for Hints 2026 Small model gets hints, not full answers
RouteProfile 2026 Graph-based profiling for cold-start routing; handles unseen models using public benchmark signals
MTRouter 2026 Cost-aware multi-turn routing via history-model joint embeddings; 58.7% cost reduction (ACL 2026)
STEER 2025 Confidence-guided stepwise routing between small/large models; no trained router
Routing, Cascades & User Choice 2026 Game-theoretic analysis: optimal routing is usually static with no cascading; exposes provider/user misalignment

Context & Inference

Paper Year Key Result
Lost in the Middle 2023 Models struggle with mid-context info
Context Rot 2025 Degradation before context limits
RAG vs Long Context 2025 Complementary strengths by query type
Self-Route Hybrid 2024 Adaptive RAG + long context
InfiniteICL 2025 90% reduction, 103% performance
YaRN Context Extension 2023 10x less tokens for context extension
SkyLadder 2025 22% training time savings
TRIM 2024 19.4% token savings on GPT-4o
ILRe 2025 Intermediate-layer retrieval cuts prefill to O(L); ~180x speedup, 1M tokens in ~30s
Context Length Alone Hurts 2025 Input length itself degrades performance even with perfect retrieval
ContextBudget (BACM) 2026 Budget-aware context management as constrained sequential decision; curriculum RL learns when/how much history to compress
LCLMs (End-to-End Context Compression) 2026 0.6B encoder compresses input blocks into latents a 4B decoder consumes directly; ~16x input compression with little accuracy loss

KV Cache & Inference

Paper Year Key Result
PagedAttention (vLLM) 2023 Near-zero KV cache waste
RadixAttention (SGLang) 2023 Auto KV cache reuse
KV Cache Survey (2026) 2026 Comprehensive techniques survey
VectorQ Semantic Caching 2025 Up to 100x latency reduction
KV-Compress 2024 Variable-head-rate compression
vAttention 2024 1.99x throughput over vLLM
LazyLLM 2024 Dynamic token pruning at prefill
SlimInfer 2025 1.88x latency reduction
Mirror Speculative Decoding 2025 Breaks serial barrier
LongSpec 2025 Constant memory speculative decoding
Speculative Speculative Decoding 2026 Parallelizes speculation+verification; 30% faster than standard SD (ICLR 2026)
IceCache 2026 Semantic clustering for KV pages; 99% accuracy at 25% token budget
Can I Buy Your KV Cache? 2026 KV cache marketplace: publishers precompute, agents load instead of prefill; 9-50x cheaper compute on Qwen3-4B
LMCache 2025 KV cache across GPU/CPU/disk/network; up to 15x throughput with vLLM
KV-Fold 2026 One-step KV-cache recurrence; training-free long-context inference
Thin Keys, Full Values 2026 SVD-based key-cache compression; up to 16x combined with GQA + quantization
Make Each Token Count 2026 Learnable retention gates for KV eviction that improve long-context accuracy
Meta-Soft 2026 Composable meta-tokens for context-preserving KV cache compression
KeepKV 2025 Adaptive lossless merging; 2x+ throughput at 10% KV budget
FreeKV 2025 Training-free speculative KV retrieval; up to 13x speedup, near-lossless
SmallKV 2025 Small-model-assisted eviction compensation; 1.75-2.56x higher throughput
Semantic Caching (Microsoft) 2025 Optimal semantic cache is NP-hard; Reverse Greedy + bandit learning
SpecFormer 2025 Lossless non-autoregressive drafting that holds up under large-batch serving
LaProx 2026 Output-aware, layer-wise KV eviction modeling attention×value interaction; beats prior eviction across 19 LongBench/NIAH datasets
Continuous Semantic Caching 2026 Theory for semantic caching in continuous embedding space; dynamic ε-net + kernel ridge regression
Learning to Draft (LTD) 2026 RL co-adapts draft+verify policies to optimize true throughput, not acceptance length (ICLR 2026)
DDTree (Block Diffusion) 2026 Block-diffusion draft tree for speculative decoding; outperforms EAGLE-3 at matched node budget
Graft 2026 Training-free prune-then-retrieve framework for speculative decoding draft trees; 5.41× speedup, 21.8% over EAGLE-3 on Qwen3-235B

Prompt Optimization

Paper Year Key Result
APE (Automatic Prompt Engineer) 2022 LLMs generate optimal prompts
Concise Chain-of-Thought 2024 48.7% shorter, negligible quality loss
Chain of Draft 2025 Only 7.6% of CoT tokens used
Semantic Compression 2023 Semantic compression with LLMs
Tokenomics 2026 Code review = 59.4% of tokens in agentic SE; input context dominates at 53.9%
IAPO 2026 Information-aware policy optimization; 36% reasoning-length reduction
SelfBudgeter 2025 Self-estimated reasoning budget via budget-guided GRPO; ~61% length cut
Step Pruner 2025 Step-aware RL reward; 33% of tokens at equal accuracy
BudgetThinker 2025 Budget-signaling control tokens for precise reasoning-length control
Extra-CoT 2026 Mixed-ratio SFT + RL for extreme-ratio CoT compression; ~73% token cut on MATH-500 with +0.6% accuracy
CROP 2026 Length-regularized automatic prompt optimization; up to ~80.6% output-token reduction (Google/Purdue)

Community Resources

Related Projects

  • LLM Safe Haven - Security toolkit for AI coding agents. npx llm-safe-haven hardens Claude Code, Cursor, Windsurf in 60 seconds. Companion project — agent retries from security failures waste tokens.
  • Awesome AI Efficiency (Pruna) - Curated list on making AI faster, cheaper, smaller, and greener.
  • Awesome Efficient LLM - Large curated list of efficient-LLM papers and tools.

Blogs

Discussions

Podcasts


Licensed under CC BY 4.0. See LICENSE.

Releases

Packages

Contributors