Add TurboQuant KV cache compression for prefix cache (4.6x)#233

Open

arozanov wants to merge 137 commits intowaybarrios:mainfrom

arozanov:feature/turboquant-kv-cache

Commits on Apr 14, 2026

Commits on Apr 16, 2026

Add TurboQuant KV cache compression for prefix cache
arozanov
committed
Fix is_trimmable regression: add duck-typing fallback for KVCache/QuantizedKVCache
arozanov
committed
Address review: fix is_trimmable fallback, use packed arrays in memory estimation
arozanov
committed
test(memory-cache): align with quantized wrapper

Thump604
authored and
arozanov
committed
Address review: fail-fast, trim warning, mutual exclusion
arozanov
committed
Fix failure for test_cancellation_does_not_release_lock_before_worker_finishes
perry2of5
committed
Fix Gemma 4 streaming reasoning parser: buffer partial markers (#355)

janhilgard
and
claude
authored
fix(api): make response_format actually work on chatty tool-calling models (#356)

janhilgard
and
claude
authored
Strip markdown code fences from streaming response_format content (#357)

janhilgard
and
claude
authored

Commits on Apr 17, 2026

Commits on Apr 18, 2026

Commits on Apr 19, 2026

[NEW FEATURE] --warm-prompts for agent cold-start TTFT + bench-serve multi-msg (#373)
waybarrios
authored

Commits on Apr 20, 2026

Commits on Apr 21, 2026

Commits on Apr 22, 2026

Commits on Apr 23, 2026

Commits on Apr 24, 2026

Commits on Apr 25, 2026

Merge upstream: tool call fixes, stream safety, SSD cache, warm prompts
arozanov
committed

Commits on Apr 30, 2026

Fix kimi tool parser: func_name extraction + remove dead code
arozanov
committed