Please add option 'Turn the depthmap into absolute black/white'

[G-force78]

Firstly thank you so much for fixing the depth map issue I was going out of my mind trying so many fixes.

This is the feature I refer to https://github.com/Extraltodeus/depthmap2mask
Its a great feature to use to create clean cut backgrounds

[graemeniedermayer]

One thing to consider is rather than using a hard cut on depth. We could use the rembg repo, https://github.com/danielgatis/rembg . it can sometimes help the depth information.

Current depth calculation

background removal before depth calculation

Reapplying the mask after depth calculation.

It can be really useful for making layered 3d scenes. I did start are extension for rembg (I haven't fully finished testing it although it might work for you). Initially I was interested in the clothing segmentation so that's why it isn't called rembg.
https://github.com/graemeniedermayer/clothseg

I could probably put together a pull request in the next few days if it's of interest?

[graemeniedermayer]

After rereading the original post, I should add that you can also turn these can also be turned into black/white masks too.

[thygate]

The next version has options to clip and expand depth range with near and far sliders, set them equal to get a b&w cut mask ..
(The 3D-Photo-Inpainting is also in the next version, no options yet, and i still need to do some more testing, it's coming soon)

The near and far clipping works on the 16 bit depth map and gives you 1000 steps on each slider to select the threshold.

Of course it's a bit cumbersome finding the threshold to cut at (0.8 is a good start) by regenerating the map constantly ..
Preferably this should be done client side in js to edit already generated depth maps, and have an option like a 'colorpicker' or histogram to more easily find the depth to cut at. I have no plans for this in the near future though ..

[thygate]

I made the commit, feature is now available.

[steampunkchris]

Quick question... are all the depth maps currently saved in 16 bit?

edit: LOL - answered my own question... Yes! They are... I am very excited about that =D

[thygate]

Is the clip option adequate to generate the b&w masks, create clean cut backgrounds ?

[G-force78]

Is the clip option adequate to generate the b&w masks, create clean cut backgrounds ?

Im not too sure I am getting another error now

Error loading script: depthmap.py
Traceback (most recent call last):
  File "/content/stable-diffusion-webui/modules/scripts.py", line 205, in load_scripts
    module = script_loading.load_module(scriptfile.path)
  File "/content/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module
    exec(compiled, module.__dict__)
  File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/depthmap.py", line 54, in <module>
    from inpaint.mesh import write_ply, read_ply, output_3d_photo
  File "/content/stable-diffusion-webui/extensions/stable-diffusion-webui-depthmap-script/scripts/inpaint/mesh.py", line 16, in <module>
    from moviepy.editor import ImageSequenceClip
  File "/usr/local/lib/python3.8/dist-packages/moviepy/editor.py", line 26, in <module>
    imageio.plugins.ffmpeg.download()
  File "/usr/local/lib/python3.8/dist-packages/imageio/plugins/ffmpeg.py", line 37, in download
    raise RuntimeError(
RuntimeError: imageio.ffmpeg.download() has been deprecated. Use 'pip install imageio-ffmpeg' instead

[thygate]

issue with imageio, please run pip install imageio==2.4.1

[thygate]

well, any progress ?

[G-force78]

It does make a nice cut however for some reason the inpainting is only picking up the outside of the mask, which could well be because of the model Im using at the moment which is a custom dreambooth one so will have to test it on the 1.5inpaint or sd 2.1. From top to bottom after the original init image is far 0 near 0.3, far 0 near 0.6, far 0 near 0.9.

[thygate]

Those look like pretty good masks ... Might benefit from a slight post blur maybe ?

I will also merge the PR by @graemeniedermayer to include the rembg asap ..

[thygate]

Merge complete, rembg now available. Would this all benefit from a small blur ?

[G-force78]

yeah the masks are good, I dont know what the blur would be for? For the immediate area near the border of the mask? I guess rembg can make a cleaner boundary