Is it possible to run this without an NVMe SSD?

[wuqs-net]

Hello,
Thanks for the great work! I'm trying to run Medusa on a server without an NVMe SSD. Could this be achieved (maybe with RAM disk) with minor modification to the code? Thanks again!

[ShaoxunZeng]

Yes, sure.
In current implementation, we store all parameters in NVMe using SPDK for higher bandwidth, you can modify the codes to use the filesystem and any devices you like in the function load_weight

Medusa/vllm/model_executor/models/llama.py

Line 403 in 6581d2e

def load_weights(self,