[Feature request] API

[semjon00]

Thanks @graemeniedermayer for suggestion.

Example: https://github.com/Mikubill/sd-webui-controlnet/blob/main/scripts/api.py

[graemeniedermayer]

I've started working on a branch for this here. https://github.com/graemeniedermayer/stable-diffusion-webui-normalmap-script/tree/depth_api .

There's still some pretty big bugs and hacky stuff.

[semjon00]

@graemeniedermayer This is great news! Please target standalone-interface branch for now, it will be easier to merge it later. By the way, have you had a chance to check out the normalmap generation (in the dev branch) I took from your repository?

[graemeniedermayer]

My most recent update is pretty independent from the main script although some of the imports might need to change.

By the way, have you had a chance to check out the normalmap generation (in the dev branch) I took from your repository?

Mostly looks great! One thing I'm not so sure about is the final normalisation step, because normals are vectors

normal = (normal - normal.min()) / (normal.max() - normal.min())  # normalize to [0; 1]
normal = np.clip(normal * 256, 0, 256 - 0.1)  # Clipping form above is needed to avoid overflowing
normal = normal.astype(np.uint8)

Any example of where this might fail if you have an normal image with all vectors of (sqrt(1/2), sqrt(1/2), 0) that will become output (1, 1, 0) which no longer has unitary vectors. So I think you'd want to do a vector-wise normalisation.

So something like

n = np.linalg.norm(normal, axis=2)
normal[:, :, 0] /= n
normal[:, :, 1] /= n
normal[:, :, 2] /= n
normal += 1
normal /= 2
normal = np.clip(normal * 256, 0, 256 - 0.1)   # Clipping form above is needed to avoid overflowing
normal = normal.astype(np.uint8)

I am also not a huge fan of the post blur so maybe we should remove it.

[semjon00]

@graemeniedermayer Right, it should normalize individual vectors/pixels, not the whole image 😅
As for post-blur, my math intuition says that this is not a terribly bad idea - but maybe I am wrong.
Force-pushed a new version, could you please give it a look?

[graemeniedermayer]

It's not that it's a terrible bad idea. Ir's just it might be redundant plus require a second normalisation. Apply multiple Gaussian blur in a row are equivalent to a single larger Gaussian blur.

https://computergraphics.stackexchange.com/questions/256/is-doing-multiple-gaussian-blurs-the-same-as-doing-one-larger-blur

But pre-blur is in depth space and the post-blur is in normal space, so it is a little non-obvious.

[semjon00]

@graemeniedermayer Aside from that, is it ok to push?

[graemeniedermayer]

Yup it's looking good!

Also with the single video there are 16 bit formats. Mostly FFV1 codec with either avi or nut as a video container (avi is a much more supported format). 16bit isn't super well supported by most video editors (blender might be able to understand 16bit avi I'll have to experiment).

https://stackoverflow.com/questions/62815543/python-16bit-grayscale-video-from-series-of-images

edit: disappointingly blender seems to down-sample 16 bit video to 8 bit for textures (16 bit still works for images sequences)

[graemeniedermayer]

Oh also should this issue be closed? because of #296

[semjon00]

@graemeniedermayer Right, closed the issue :)
Just pushed normalmap generation into the main: 74dc96b. Now the job of migrating users to this repository is almost completed (#272). Once they hop into this repo, they will get some extra goods (like generating normalmaps with zoedepth). This could be accelerated a bit. @graemeniedermayer Could you please change the NormalMap repository so it redirects users to this repository? Otherwise they might not find out that this repo allows generating normalmaps, too. Archiving would also help.

[G-force78]

This is great just what I was looking for thank you. Is there a requirements.txt anywhere to ensure the right dependencies are installed?

[semjon00]

🙂
If this is installed as a WebUI script, then all the needed dependencies will be installed on the first WebUI run, into the corresponding venv.

[G-force78]

🙂 If this is installed as a WebUI script, then all the needed dependencies will be installed on the first WebUI run, into the corresponding venv.

No I use it as a standalone app as google block it on free tier. I think i sorted it anyway thanks.