Textual Inversion / Dreambooth / ControlNet / LoRA for Kandinsky

[patrickvonplaten]

Thanks to @yiyixuxu Kandinsky 2.1: #3308 is merged to diffusers.

Anybody interesting in giving dreamboothing/textual inversioning the model?

[knoopx]

checking this myself now, here's a reference implementation https://github.com/TheDenk/Kandinsky-2-textual-inversion/blob/main/notebooks/train_dreamboth.ipynb

[freespirit]

I’d like to try adding LoRA. I don’t have experience yet with Dreambooth and Textual Inversion so I’d prefer to focus on LoRA only.

I’ve used LoRA for fine-tuning and inference with StableDiffusion so I’d use that as reference. But I still may need some guidance.

If this sounds good I’d be happy to start planning it.
@knoopx is LoRA in scope for you?

[knoopx]

@freespirit nope but the training code should be pretty much the same (with the addition of LoRa layers), the hard part will be inference.

[mohammedElfatihSalah]

@patrickvonplaten I would love to help in this

[sayakpaul]

Kandinsky extensions for TI, DreamBooth, etc. didn't garner enough interest from the community, sadly. So, will close.