Skip to content

[bug fix] advanced dreambooth lora sdxl - fixes bugs described in #6486 #6599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jan 17, 2024
Merged

[bug fix] advanced dreambooth lora sdxl - fixes bugs described in #6486 #6599

merged 4 commits into from
Jan 17, 2024

Conversation

linoytsaban
Copy link
Collaborator

fixes the 3 issues with the advanced dreambooth lora sdxl training script, as described in #6486:

  1. redundant retraction of embeddings when performing pivotal tuning
  2. zeroing of text encoder params (when pivoting halfway we only optimize the embeddings during the first X% of steps - to handle correctly (accelerate wise) the param change midway for the optimizer, we set the relevant params to zero) only happens for one step
  3. incorrect casting of text encoder params to float32

@linoytsaban
fixes bugs:
70ebdf3 
1. redundant retraction
2. param clone
3. stopping optimization of text encoder params
@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@linoytsaban linoytsaban marked this pull request as ready for review January 16, 2024 18:36
@linoytsaban linoytsaban self-assigned this Jan 17, 2024
sayakpaul
sayakpaul approved these changes Jan 17, 2024
View reviewed changes
Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge away!

Additionally, it'd be nice to put together a readme for the advanced diffusion training directory. This is because many of our users refer to them directly.

@sayakpaul sayakpaul merged commit 619e3ab into huggingface:main Jan 17, 2024
@linoytsaban
Copy link
Collaborator Author

definitely! Thanks! ❤️

@linoytsaban linoytsaban mentioned this pull request Jan 17, 2024
Closed
@linoytsaban linoytsaban deleted the advanced_script_fixes branch February 8, 2024 17:07
AmericanPresidentJimmyCarter pushed a commit to AmericanPresidentJimmyCarter/diffusers that referenced this pull request Apr 26, 2024
@linoytsaban
[bug fix] advanced dreambooth lora sdxl - fixes bugs described in hug…
72a5ac9 
…gingface#6486 (huggingface#6599)

* fixes bugs:
1. redundant retraction
2. param clone
3. stopping optimization of text encoder params

* param upscaling

* style
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sayakpaul sayakpaul sayakpaul approved these changes

Assignees

@linoytsaban linoytsaban

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@linoytsaban @HuggingFaceDocBuilderDev @sayakpaul