img2img is broken

[vmobilis]

Reference image:

Command line:
./sd --model "dreamshaper_8.safetensors" -o "img2img.png" -p "green apple" --mode img2img -i "horse.jpg" --strength 0.4 --type f16

2eac844	cc92a6a	8f4ab9a

Result is almost or identical to reference.
F32 and quantized types work same.
Regression is introduced in 8f4ab9a commit.

Model version is Stable Diffusion 1.5.
https://civitai.com/models/4384?modelVersionId=128713

[stduhpf]

I can't reproduce this issue with commit 19d876e

8f4ab9a also works for me:

I don't know how this could be hapenning on your end, all I can think is that maybe it runs in inpaint mode with an empty mask? But this shouldn't be possible, if the mask is not specified, it's supposed to be initialized to fill the whole frame.

[vmobilis]

@stduhpf, I found the reason: compiler filled mask with zeroes.
If I printed few values from arr, the mask was filled with 0xFF, but otherwise the buffer seemed to be created without fill.
Depending on where I tried to copy data from arr to mask_image, I either got zeroed mask or segmentation fault.

By trials I ended up with #610.

[leejet]

There is a dangling pointer issue here, arr memory is released after going out of scope, but it is still referenced by mask_image_buffer. I fixed this issue in master, please try it if it can solve your problem.

[vmobilis]

@leejet, moving arr[] to scope of mask_image_buffer fixed the issue, thank you!

@leejet, @stduhpf, will it be acceptable to move mask creation completely to img2img part?
This will save few megabytes for txt2img generation.
Like #612.

[Disonantemus]

Seems wrong to me, am I missing something?

Reference image, generated with:

magick logo: -crop 512x512+50+0 -gravity Center -background white -extent 512x512 img2img.png

img2img.png:

Command to generate images with strengths from 0.0 to 1.0:

for i in 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 ; do sd --model /run/media/user/drive/models/dreamshaper_8.safetensors -p "green apple" --mode img2img -i img2img.png --strength "$i" --steps 5 -o output"$i".png ; done

Outcome:

• 0.0 and 0.1 are the same
• 0.2 and 0.3 are the same
• 0.4 and 0.5 are the same
• 0.6 and 0.7 are the same
• 0.8 and 0.9 and 1.0 are the same

To me: every strength value should be different, am I wrong?

Output images:

output0.0.png	output0.1.png

output0.2.png	output0.3.png

output0.4.png	output0.5.png

output0.6.png	output0.7.png

output0.8.png	output0.9.png	output1.0.png

[stduhpf]

@Disonantemus What you seem to be missing is that in img2img mode, the strength correspond to the percentage of denoising steps that actually need to be performed. With a total of 5 steps, if the strength is set between 0.2 and 0.4, only one step will be done. (And 0 steps for under 0.2, 2 steps between 0.4 and 0.6, and so on.)

Maybe it would make sense to change it so it actually does all the steps to ask for and change the max number of steps accordingly, but I believe it's done this way to be consistent with the way Auto1111 is doing it.

[Disonantemus]

I understand.

I was trying to change style of an image using img2img, from here, but the outcome is not very good.

I'm gonna try using control net, but I can't find ip adapter.

What is possible to do with stable diffusion to change style from a personal photo, allowing to recognize the faces of the people?

If isn't possible, there is another alternative to do this?