FreeInit

[a-r-r-o-w]

Model/Pipeline/Scheduler description

FreeInit allows for improved temporal consistency of diffusion-based video models (for example, AnimateDiff and SVD). With no additional training requirement, it produces much higher quality videos.

Going through the code, it seems like an easy addition to current diffusers, and the results
presented, as well as those from initial testing, are awesome!

Youtube

Results
Baseline	Baseline + FreeInit

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

Paper
Project Page
Code

Authors: @TianxingWu, @ChenyangSi, @yumingj, @ziqihuangg, @liuziwei7

Thank you for the amazing work in this, FreeU, VideoBooth and many others! It has been very informative and motivating to learn from your work.

@sayakpaul @patrickvonplaten

[sayakpaul]

Seems like a really cool addition. @DN6 WDYT?

[DN6]

Yeah I'm cool with adding to the AnimateDiff. Seems like the changes would be minimal.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.