Is concurrent-request batching available to LM Studio's Splash runtime?

[hugheba]

Hi! Standalone Splash 1.3.1 batches concurrent requests together with DFlash 2 speculation really well. On an M5 Pro (64 GB) with mlx-community/Qwen3.6-35B-A3B-4bit, 4 concurrent requests all finish in about 4.9 s, roughly 243 tok/s combined.

The Splash engine bundled in LM Studio (runtime splash-mac-arm64-apple-metal-advsimd@0.0.5, LM Studio 0.4.26) serves the same model one request at a time: 4 concurrent requests finish at about 3 / 6 / 9 / 13 s, roughly 94 tok/s combined. I've reported this to LM Studio here: lmstudio-ai/lmstudio-bug-tracker#2490

Is batching in the engine version shipped to LM Studio and just not enabled by their integration? Or is it a newer engine feature that LM Studio hasn't picked up yet? A rough answer would help me decide whether to stay on standalone Splash or move back to LM Studio.

Thanks for the engine. The speedup on a laptop is remarkable.

[jianc99]

Hi, batching was supported from the first version of Splash. Not sure why LM Studio disables it. Will coordinate with them about this!