WARNING 02-09 10:24:45 cpu.py:99] ray is not supported on CPU, fallback to mp distributed executor backend.

[jdmedeiros]

I am trying to run vLLM across 9 r5.large (2 vCPU x 16 GB) instances on AWS. I compiled vLLM from source as indicated here https://docs.vllm.ai/en/stable/getting_started/installation/cpu/index.html and I am wondering if this (ray is not supported on CPU) is configurable or if it's by design.

[aliciasoliveiraa]

Hello @jdmedeiros, is there a solution to this problem?

[jdmedeiros]

At the time I could not get it to work with CPU only; I have not tried recently. I went a different route using https://github.com/exo-explore/exo and https://tailscale.com/

Essentially, create a free account here https://tailscale.com/ and generate an API token. Install tailscale and exo on all your devices and then run exo as:

exo --discovery-module tailscale --tailscale-api-key tskey-api-XXXXCNTRL-XXXXX --tailnet-name [[email protected]](mailto:[email protected])

[email protected] is your account in Tailscale

That will allow you to achieve basically the same thing.