Skip to content

Add Tutorial of QAT with X86InductorQuantizer #2717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Jan 24, 2024
Merged

Add Tutorial of QAT with X86InductorQuantizer #2717

merged 10 commits into from
Jan 24, 2024

Conversation

leslie-fang-intel
Copy link
Contributor

@leslie-fang-intel leslie-fang-intel commented Dec 19, 2023 •
edited
Loading

Description

This diff adds the new tutorial of PyTorch 2 Export Quantization-Aware Training (QAT) with X86 Backend through Inductor.

Checklist

  • The issue that is being fixed is referred in the description (see above "Fixes #ISSUE_NUMBER")
  • Only one issue is addressed in this pull request
  • Labels from the issue that this PR is fixing are added to this pull request
  • No unnecessary issues are included into this pull request.

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Dec 19, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/2717

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 9f307f7 with merge base d9e986f (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@leslie-fang-intel
Copy link
Contributor Author

cc @jgong5 @jerryzh168 @svekars please help to take a look of this PR.

@svekars svekars added the 2.2 label Dec 19, 2023
svekars
svekars reviewed Dec 19, 2023
View reviewed changes
Copy link
Contributor

@svekars svekars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @leslie-fang-intel! Just a couple editorial suggestions.

@jerryzh168
Copy link
Contributor

should we incorporate this into the PTQ tutorial to reduce the number of tutorials?

Maybe we should do the same for XNNPACKQuantizer as well, cc @andrewor14

svekars
svekars reviewed Dec 19, 2023
View reviewed changes
Copy link
Contributor

@svekars svekars left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, please add to https://github.com/pytorch/tutorials/blob/main/prototype_source/prototype_index.rst

@andrewor14
Copy link
Contributor

andrewor14 commented Dec 20, 2023 •
edited
Loading

Hi @leslie-fang-intel, thanks for working on this. I agree with @jerryzh168 that we should merge this with the X86Inductor PTQ tutorial. What you're adding here seems very similar to the existing PT2 export QAT tutorial except for maybe these 2 lines:

  quantizer = X86InductorQuantizer()
  quantizer.set_global(xiq.get_default_x86_inductor_quantization_config(is_qat=True))

so I feel it's not worth making a brand new tutorial just for this use case. What do you think?

@leslie-fang-intel
Copy link
Contributor Author

@jerryzh168 @andrewor14 Thanks for the comment. Merge the PTQ and QAT for X86InductorQuantizer into one tutorial. Please kindly help to take a look again.

@leslie-fang-intel
Copy link
Contributor Author

@svekars I am afraid the changes in this PR may have conflict with #2716. Should we merge these 2 PRs together to facilitate the review or merge process? Or do you prefer to merge #2716 firstly then we rebase current PR after it.

@svekars
Copy link
Contributor

svekars commented Dec 21, 2023

@svekars I am afraid the changes in this PR may have conflict with #2716. Should we merge these 2 PRs together to facilitate the review or merge process? Or do you prefer to merge #2716 firstly then we rebase current PR after it.

I'm good either way, if you have bandwidth you can merge these two in one PR and close the one we don't need to merge. But also happy with the rebase plan.

jerryzh168
jerryzh168 reviewed Dec 21, 2023
View reviewed changes
jerryzh168
jerryzh168 reviewed Dec 21, 2023
View reviewed changes
prototype_source/pt2e_quant_x86_inductor.rst Outdated
@@ -154,6 +176,7 @@ Finally, we will convert the calibrated Model to a quantized Model. ``convert_pt
::

converted_model = convert_pt2e(prepared_model)
torch.ao.quantization.move_exported_model_to_eval(converted_model)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this needed for both PTQ and QAT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we could also have a separate section for QAT, like what we did in https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html#quantization-aware-training

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment. Yean, I think it will be much clearer for reader. Move the QAT into a separate section. Please kindly help to take a look again @jerryzh168.

@leslie-fang-intel leslie-fang-intel mentioned this pull request Dec 22, 2023
Merged
4 tasks
@leslie-fang-intel leslie-fang-intel force-pushed the leslie/add_qat_tutorial_in_x86_inductor_quantization branch from 0eb4ae7 to 16aa287 Compare January 12, 2024 08:35
@leslie-fang-intel
Copy link
Contributor Author

Hi @svekars @jerryzh168, I have rebased to fix the conflict. Please kindly help to take a look again.

@svekars svekars merged commit eec8d56 into pytorch:main Jan 24, 2024
@leslie-fang-intel leslie-fang-intel mentioned this pull request Jan 25, 2024
Closed
4 tasks
HDCharles pushed a commit that referenced this pull request Jan 26, 2024
@leslie-fang-intel @svekars @HDCharles
Add Tutorial of QAT with X86InductorQuantizer (#2717)
3595dd3 
* Add QAT Tutorial
---------

Co-authored-by: Svetlana Karslioglu <svekars@meta.com>
svekars added a commit that referenced this pull request Feb 2, 2024
@leslie-fang-intel @svekars
Add Tutorial of QAT with X86InductorQuantizer (#2717)
8ebf392 
* Add QAT Tutorial
---------

Co-authored-by: Svetlana Karslioglu <svekars@meta.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@svekars svekars svekars approved these changes

@jgong5 jgong5 jgong5 approved these changes

@jerryzh168 jerryzh168 Awaiting requested review from jerryzh168

Assignees
No one assigned
Labels
2.2 cla signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@leslie-fang-intel @jerryzh168 @andrewor14 @svekars @jgong5 @facebook-github-bot