Partial diffusion for the stable diffusion pipelines

[gunshi]

Hey!
I was searching the issues page for an answer for whether we can currently do inference for any pipeline where we want to start from an intermediate timestep (say I add noise to an original image corresponding to timestep 200, then I want to be able to get the SD model's output on this noised image by telling it that it should assume it is currently starting from timestep 800).

As far as I can tell this I currently not possible in the pipeline_stable_diffusion.py since it calls the scheduler.set_timesteps() which itself constructs the time steps array by spacing out the 0-1000 range (for SD) into chunks of num_inference_steps intervals. There's also a couple other functions within the scheduler class which assume we will start from complete noise, and use the self.config.num_train_timesteps variable to do computation.
This proposal seems related to this feature but is specific to SDXL, am I correct?
Would it be possible to add partial diffusion support to more pipelines and if not could you point me to code that I could refer to to make analogous changes locally on my end to enable this for SD?
Thanks for any pointers!

[sayakpaul]

Cc: @patrickvonplaten

Hi.

You can refer to the following PR that contributed the support for it to SDXL: #4015.

You can also refer to the docs that gives you a nice overview of the recommendations to be aware of when using this approach for SDXL.

Hope that helps.

[gunshi]

thank you!