triton fails when NO_GPU and NO_CUDA installed and performing generation using ONLY_CPU #1499
Unanswered
Davros666
asked this question in
CATCH-ALL: alpha testing the `multi-backend-refactor`
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When using just Intel CPUs on a rack server with NO_GPU and NO_CUDA nonsense, the triton code fails with;
File "/home/davros666/flux/.venv/lib/python3.12/site-packages/triton/runtime/driver.py", line 8, in _create_driver raise RuntimeError(f"{len(actives)} active drivers ({actives}). There should only be one.") RuntimeError: 0 active drivers ([]). There should only be one.
Luckily there's an easy fix.
Simply change /host/your_folder/.your_venv/lib/python3.12/site-packages/bitsandbytes/triton/triton_utils.py to;
import importlib
def is_triton_available():
#return importlib.util.find_spec("triton") is not None
return False
and the tedious work-halting problem goes away instantly.
Maybe one day the code will be written to cater to such an option, namely not having any nvidia drivers installed but this is something that is being made compulsory now under new UN international laws under threat of fines or execution by beheading one hears.
Beta Was this translation helpful? Give feedback.
All reactions