Skip to content

Add oneAPI device selector for xpu and some other changes. #6112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Dec 23, 2024
Merged

Add oneAPI device selector for xpu and some other changes. #6112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
afde678
Add oneAPI device selector and some other minor changes.
simonlui Dec 19, 2024
fdccaa1
Fix device selector variable name.
simonlui Dec 19, 2024
b588fdd
Flip minor version check sign.
simonlui Dec 19, 2024
895e78e
Undo changes to README.md.
simonlui Dec 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion comfy/cli_args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,8 @@ def __call__(self, parser, namespace, values, option_string=None):

parser.add_argument("--directml", type=int, nargs="?", metavar="DIRECTML_DEVICE", const=-1, help="Use torch-directml.")

parser.add_argument("--disable-ipex-optimize", action="store_true", help="Disables ipex.optimize when loading models with Intel GPUs.")
parser.add_argument("--oneapi-device-selector", type=str, default=None, metavar="SELECTOR_STRING", help="Sets the oneAPI device(s) this instance will use.")
parser.add_argument("--disable-ipex-optimize", action="store_true", help="Disables ipex.optimize default when loading models with Intel's Extension for Pytorch.")

class LatentPreviewMethod(enum.Enum):
NoPreviews = "none"
Expand Down
6 changes: 4 additions & 2 deletions comfy/model_management.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ class CPUState(Enum):
try:
import intel_extension_for_pytorch as ipex
_ = torch.xpu.device_count()
xpu_available = torch.xpu.is_available()
xpu_available = xpu_available or torch.xpu.is_available()
except:
xpu_available = xpu_available or (hasattr(torch, "xpu") and torch.xpu.is_available())

Expand Down Expand Up @@ -214,12 +214,14 @@ def is_nvidia():
if args.cpu_vae:
VAE_DTYPES = [torch.float32]


if ENABLE_PYTORCH_ATTENTION:
torch.backends.cuda.enable_math_sdp(True)
torch.backends.cuda.enable_flash_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(True)

if int(torch_version[0]) == 2 and int(torch_version[2]) >= 5:
torch.backends.cuda.allow_fp16_bf16_reduction_math_sdp(True)
Copy link
Owner

@comfyanonymous comfyanonymous Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no situations where the math backend is actually used by ComfyUI unless you force it.

Copy link
Contributor Author

@simonlui simonlui Dec 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should've explained this better. For any non-Nvidia GPUs, the math backend is what ends up being used if Pytorch Attention is selected since the Flash Attention and mem efficient backends are CUDA only, with AMD's implementations for both only making it around 1-2 weeks ago in the nightly packages. I can try and gate this off a bit better if you want, but the Pytorch change mentioned does slow things down for GPUs that are stuck in that situation like Intel right now.


if args.lowvram:
set_vram_to = VRAMState.LOW_VRAM
lowvram_available = True
Expand Down
4 changes: 4 additions & 0 deletions main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,10 @@ def execute_script(script_path):
os.environ['HIP_VISIBLE_DEVICES'] = str(args.cuda_device)
logging.info("Set cuda device to: {}".format(args.cuda_device))

if args.oneapi_device_selector is not None:
os.environ['ONEAPI_DEVICE_SELECTOR'] = args.oneapi_device_selector
logging.info("Set oneapi device selector to: {}".format(args.oneapi_device_selector))

if args.deterministic:
if 'CUBLAS_WORKSPACE_CONFIG' not in os.environ:
os.environ['CUBLAS_WORKSPACE_CONFIG'] = ":4096:8"
Expand Down
Loading