[Tracker] modularize inferencing during and after training in the example scripts

[sayakpaul]

We provide support for running validation inference during and after training in our officially maintained training examples. This is very helpful to keep track of the training progress.

We could modularize some bits in the example to reduce the LoC.

The train_lcm_distill_lora_sdxl.py script already does this:

diffusers/examples/consistency_distillation/train_lcm_distill_lora_sdxl.py

Line 100 in 33d2b5b

def log_validation(vae, args, accelerator, weight_dtype, step, unet=None, is_final_validation=False):

It would be nice to follow something similar for the rest of the scripts too. Here's a handy list of the scripts where we'd like to incorporate this change:

• DreamBooth
  • SD LoRA
  • SDXL LoRA
• InstructPix2Pix
  • SD
  • SDXL
• Text-to-image
  • SD
  • SD LoRA
  • SDXL LoRA

Feel free to comment here if you're interested.

Also, when opening PRs, please target one example at a time and please tag me in the PRs.

[charchit7]

Hey @sayakpaul for text-to-image, SD
log-validation is present :

diffusers/examples/text_to_image/train_text_to_image.py

Line 141 in 33d2b5b

def log_validation(vae, text_encoder, tokenizer, unet, args, accelerator, weight_dtype, epoch):

haven't gone through the code, will check more if we need to modify it.

[sayakpaul]

Ideally, this should be delegated to log_validation() too:

diffusers/examples/text_to_image/train_text_to_image.py

Line 1047 in 33d2b5b

# Run a final round of inference.

As done in train_lcm_distill_lora_sdxl.py.

[charchit7]

Got it. @sayakpaul I missed that part. Thanks!

[sangyeon-k]

Hey @sayakpaul, @charchit7
I'm interested in incorporating the change to InstructPix2Pix SD

[coolyashas]

Hey @sayakpaul, @charchit7 I'm interested in incorporating the change to InstructPix2Pix SD

Hey @sang-k, I have been looking into InstructPix2Pix SD too. Would be happy to collaborate if necessary.

[sangyeon-k]

@coolyashas
Oh, I see.
I will look into InstructPix2Pix SDXL then!

[coolyashas]

Working on InstructPix2Pix SD