Autotune inference#4131

Draft

tdene wants to merge 16 commits intoNVIDIA:mainfrom

tdene:tde/autotune_inference

Commits on Apr 3, 2026

Static autotuning
tdene
committed
Dynamic mamba auto-tuning
tdene
committed
Dynamic max_requests / max_tokens tuning
tdene
committed
Add timing logs
tdene
committed
Fix buffer_size_gb
tdene
committed
Cleanup
tdene
committed
tmp
tdene
committed
Nested tags
tdene
committed
Fix typo
tdene
committed
tmp
tdene
committed
fix
tdene
committed
Untangle
tdene
committed
tmp
tdene
committed
tmp
tdene
committed
debug
tdene
committed
Two-phase
tdene
committed