fix: run BatchedEngine MLLM on dedicated MLXWorkerThread to prevent cross-thread stream errors#479

Open

xykong wants to merge 4 commits intowaybarrios:mainfrom

xykong:fix/batched-engine-stream-thread

Commits on May 1, 2026

fix: run MLLM model load and inference on dedicated MLXWorkerThread
xykong
committed
perf: split preprocessing into parallel tokenization + serial array materialization
xykong
committed
fix: MLLM prepare_for_start and prefix cache bypass for vision
xykong
committed
fix(tool_parser): handle single-quoted strings from quantized models
xykong
committed