Adding support for ControlNet pipeline for img2img flows

[mikegarts]

It would be nice to achieve a somewhat feature parity with automatic1111 in terms using controlnet with img2img flows.
Current ControlNet pipeline doesn't allow an input source image (if I understand correctly the image parameter goes to controlnet as condition and not as src image for diffusion).

I think I might try and implement this myself, but wondering how this could be achieved best?

• To add an optional src_image parameter to the ControlNet pipeline?
• To create a separate file with ControlNetImg2Img?
• Something else?

@takuma104 - since you implemented the StableDiffusionControlNetPipeline what are your thoughts?

[takuma104]

Hi @mikegarts , How about #2561 that was opened a few hours ago?

[realimposter]

@takuma104 looks like that pipeline is for inpainting + img2img, any idea how to modify it just for img2img?

[mikegarts]

Indeed #2561 is for inpainting models :)
But I think that I might be able to write something similar for img2img and submit it as a community pipeline.

[pd-brian-mccrindle]

@mikegarts
I'm also interested in getting ControlNet working with just Img2Img. If you have a branch implementing this, I can try and support where I can

[mikegarts]

@pd-brian-mccrindle Thanks lot, I will run some tests and open a pr later today.

[mikegarts]

@pedrogengo here it is: #2584