[Model Card] standardize dreambooth model card #6729
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
@sayakpaul thanks for the mention! Direct use will be used for a code example showing how to inference with diffusers? |
|
No, those will have to be changed by the model developer. We don't automatically generate "direct use" for all model cards. I think it's okay not to and edit them manually. But I don't think it should be a strict requirement. This is because for ControlNet training it's different direct use, for DreamBooth training, it's different direct use. Hope that make sense. |
|
@patrickvonplaten I am not a big fan of model_card = load_or_create_model_card(
repo_id_or_path=repo_id,
license="creativeml-openrail-m",
base_model=base_model,
instance_prompt=prompt,
model_description=model_description,
inference=True,
)Should we make everything explicit in the signature? Do we really need to? Or maybe we could just validate with a decorator and keep the kwargs specific to the training script only. I think the second one is a more lenient and cleaner approach. Once we settle on that, I will update the example scripts where we create model cards. For the ones, that don't, they can be done in a separate PR, preferably by the community. |
There was a problem hiding this comment.
the code looks good to me once @patrickvonplaten 's comments are addressed
Why did we decide to use the default model card template? this https://huggingface.co/sayakpaul/test-model-card-template-dreambooth doesn't look very nice with so many empty fields
Well my questions need to be addressed first :D #6729 (comment)
It’s the standard one I followed following trainers from other libraries. I think we can iterate on the template in future PRs. Ccing @younesbelkada to check if he has anything to suggest from the TRL world. |
ahhh you're absolutely right!! sorry I misread your code🙈 I think we should decide on a template in this PR, though. The goal here is to standardize model cards, and the template is a super important part of it, no? |
|
You are right. I will get to the changes. |
Indeed that's what we use by default in transformers too ! If you prefer you could also set an empty model card instead of using a template |
Do you have a reference? |
|
@yiyixuxu WDYT about the recent changes? :) Example: https://huggingface.co/sayakpaul/test-model-card-template-dreambooth. |
|
@sayakpaul |
|
I am now going to propagate the changes to the rest of the scripts and will ask for a review once done. Thanks! |
|
SDXL looks like so: https://huggingface.co/sayakpaul/test-sdxl-lora-dreambooth-model-card. |
| base_model=str, | ||
| base_model: str = None, | ||
| train_text_encoder=False, | ||
| instance_prompt=str, | ||
| validation_prompt=str, | ||
| instance_prompt=None, | ||
| validation_prompt=None, |
There was a problem hiding this comment.
Was having compulsion disorder :3 So, decided to fix these.
There was a problem hiding this comment.
I would even update to this 😄
base_model: Optional[str] = None,
train_text_encoder: bool = False,
instance_prompt: Optional[str] = None,
validation_prompt: Optional[str] = None,
|
@yiyixuxu this is up for another review. I decided to standardize the DreamBooth scripts and the Custom Diffusion script. Will open the rest for the community. WDYT? |
There was a problem hiding this comment.
Thanks for working on that @sayakpaul! Looks good to me :) Added a few nit comments + suggestion to add the <Gallery/> component to nicely display examples in the model card.
| base_model=str, | ||
| base_model: str = None, | ||
| train_text_encoder=False, | ||
| instance_prompt=str, | ||
| validation_prompt=str, | ||
| instance_prompt=None, | ||
| validation_prompt=None, |
There was a problem hiding this comment.
I would even update to this 😄
base_model: Optional[str] = None,
train_text_encoder: bool = False,
instance_prompt: Optional[str] = None,
validation_prompt: Optional[str] = None,| widget=widget_str, | ||
| ), | ||
| template_path=MODEL_CARD_TEMPLATE_PATH, | ||
| model_description=model_description, |
There was a problem hiding this comment.
If widget is populated, I would automatically add a </Gallery> tag to the model card description like done in https://huggingface.co/Pclanglais/Mickey-1928/blob/main/README.md?code=true. It will nicely render example images in a gallery (see here) and you wouldn't have to manually build a img_str in your notebook example. What do you think? :)
There was a problem hiding this comment.
(maybe only if widget is not none and "<gallery>/" not in model_description)
There was a problem hiding this comment.
That means the existing save_model_card will have to be rewritten a bit to support compatibility with </Gallery>. I propose not to do that in this PR as it differs in scope.
these [to-do] fields are intended to be filled manually by users, right? |
* feat: standarize model card creation for dreambooth training. * correct 'inference * remove comments. * take component out of kwargs * style * add: card template to have a leaner description. * widget support. * propagate changes to train_dreambooth_lora * propagate changes to custom diffusion * make widget properly type-annotated
What does this PR do?
An attempt to partially fix #5667 by starting to standardize the creation of model card in the
train_dreambooth.pyscript. An immediate follow-up of #6678 as well. @linoytsaban FYI as well.I have taken the liberty to fix some type annotations as well.
Generated model card: https://huggingface.co/sayakpaul/test-model-card-template-dreambooth
Notebook to test: https://huggingface.co/sayakpaul/test-model-card-template-dreambooth/blob/main/test_dreambooth_model_card.ipynb (hosted on Hub and I demand a gift for using the Hub to host the notebook :v)