FLUX.1-Fill-dev running problem

[pique2233]

Describe the bug

I don't know why, but the loading has remained stationary at this step. I'm running it on a 4090-24G graphics card. But it seems that it has been stuck since loading the pipe
(worldgen) lqz27@rise:~/.../WorldGen/flux$ python flux.py
Loading checkpoint shards: 100%|█████████████████| 2/2 [00:00<00:00, 149.45it/s]
Loading checkpoint shards: 100%|██████████████████| 3/3 [00:00<00:00, 95.33it/s]
Loading pipeline components...: 57%|███████▍ | 4/7 [00:00<00:00, 36.35it/s]You set add_prefix_space. The tokenizer needs to be converted from the slow tokenizers
Loading pipeline components...: 100%|█████████████| 7/7 [00:00<00:00, 26.22it/s]

Reproduction

import torch
from diffusers import FluxFillPipeline
from diffusers.utils import load_image

image = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup.png")
mask = load_image("https://huggingface.co/datasets/diffusers/diffusers-images-docs/resolve/main/cup_mask.png")

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("cuda")
image = pipe(
prompt="a white paper cup",
image=image,
mask_image=mask,
height=1632,
width=1232,
guidance_scale=30,
num_inference_steps=50,
max_sequence_length=512,
generator=torch.Generator("cpu").manual_seed(0)
).images[0]
image.save(f"flux-fill-dev.png")

Logs

System Info

The latest diffusers ，python=3.11

Who can help?

No response

[asomoza]

Hi, you're not giving your environment, but I'm guessing you're using windows, Flux Fill doesn't fit in a 24GB GPU if you're using some of the VRAM for the OS, I don't recall correctly but this is for the base Flux, probably Flux Fill needs more VRAM.

What you think is stuck on loading the pipe is the model loading and doing inference really slow on the CPU, you will need to use an offloading strategy or quantization.

[pique2233]

Hi, you're not giving your environment, but I'm guessing you're using windows, Flux Fill doesn't fit in a 24GB GPU if you're using some of the VRAM for the OS, I don't recall correctly but this is for the base Flux, probably Flux Fill needs more VRAM.

What you think is stuck on loading the pipe is the model loading and doing inference really slow on the CPU, you will need to use an offloading strategy or quantization.

My OS is Ubuntu20.04,CUDA version = 12.4,torch = 2.4.0,python=3.11 diffusers = 0.34.0,transformers = 4.53.2,When I checked with nvidia-smi, I found that there was no inference on the gpu, but it was still the problem from yesterday. The pipeline components had been loaded to 100%

[pique2233]

Hi, you're not giving your environment, but I'm guessing you're using windows, Flux Fill doesn't fit in a 24GB GPU if you're using some of the VRAM for the OS, I don't recall correctly but this is for the base Flux, probably Flux Fill needs more VRAM.

What you think is stuck on loading the pipe is the model loading and doing inference really slow on the CPU, you will need to use an offloading strategy or quantization.

After testing, I found that I would get stuck at this step: Pipe.to("cuda") or.to("cuda")

[asomoza]

can you run diffusers-cli env and post the output?

if this works:

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16)

but this gets "stuck"

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("cuda")

means that there's something wrong when passing the model to the GPU, if there isn't an error or the running python script doesn't break, it means that it's loading but incredible slow, probably on the CPU.

Also you're using really old versions of torch and cuda, torch is already in version 2.7.1 and cuda in 12.8, so I suggest you update them before anything else, also ensure that the version of torch you install is compiled with cuda.

But more important, what you're trying to fix will jut give you an OOM since as I told you, flux loaded in bfloat16 can't be run with a 24GB GPU if you're also using it for the GUI of your SO. With Windows you will get slow speeds but with linux you will get OOM.

[pique2233]

您可以运行diffusers-cli env并发布输出吗？

如果有效：

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16)

但这被“卡住了”

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("cuda")

意味着将模型传递给 GPU 时出现了问题，如果没有错误或者正在运行的 python 脚本没有中断，则意味着它正在加载但速度非常慢，可能是在 CPU 上。

另外，您使用的torch和版本非常旧cuda，torch 已经是 2.7.1 版本，而 cuda 是 12.8 版本，因此我建议您在执行任何其他操作之前先更新它们，同时确保您安装的 torch 版本是使用 cuda 编译的。

但更重要的是，你试图修复的问题会导致 OOM，因为正如我之前所说，如果你还用 24GB 的 GPU 来运行 SO 的 GUI，那么以 bfloat16 格式加载的 flux 就无法运行。在 Windows 上速度会很慢，但在 Linux 上会 OOM。
its my diffusers-cli env

diffusers-cli env

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- 珞 Diffusers version: 0.34.0
- Platform: Linux-5.15.0-139-generic-x86_64-with-glibc2.31
- Running on Google Colab?: No
- Python version: 3.11.13
- PyTorch version (GPU?): 2.7.1+cu128 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.33.4
- Transformers version: 4.53.2
- Accelerate version: 1.8.1
- PEFT version: not installed
- Bitsandbytes version: not installed
- Safetensors version: 0.5.3
- xFormers version: not installed
- Accelerator: NVIDIA GeForce RTX 4090, 24564 MiB
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

(pano) lqz27@rise:~/.../WorldGen/flux$

[pique2233]

can you run diffusers-cli env and post the output?

if this works:

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16)

but this gets "stuck"

pipe = FluxFillPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16).to("cuda")

means that there's something wrong when passing the model to the GPU, if there isn't an error or the running python script doesn't break, it means that it's loading but incredible slow, probably on the CPU.

Also you're using really old versions of torch and cuda, torch is already in version 2.7.1 and cuda in 12.8, so I suggest you update them before anything else, also ensure that the version of torch you install is compiled with cuda.

But more important, what you're trying to fix will jut give you an OOM since as I told you, flux loaded in bfloat16 can't be run with a 24GB GPU if you're also using it for the GUI of your SO. With Windows you will get slow speeds but with linux you will get OOM.

and i made a new conda env,now pytorch == 2.7.1,cuda=12.8 python==3.11,but it still cannot move to GPU

[asomoza]

that's weird, everything looks ok, the last question I have, how much RAM do you have?