Clarify post training quantization behaviour #2778
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/2778
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 3d04f5f with merge base 630c2e2 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Hi @jmarintur! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
recipes_source/quantization.rst
Outdated
| @@ -81,7 +81,7 @@ The full documentation of the `quantize_dynamic` API call is `here <https://pyto | |||
| 3. Post Training Static Quantization | |||
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |||
|
|
|||
| This method converts both the weights and the activations to 8-bit integers beforehand so there won't be on-the-fly conversion on the activations during the inference, as the dynamic quantization does, hence improving the performance significantly. | |||
| This method converts both the weights and the activations to 8-bit integers beforehand so there won’t be on-the-fly conversion on the activations during the inference, as the dynamic quantization does. While post-training static quantization can significantly enhance inference speed and reduce model size, this method may degrade the original model's performance. | |||
There was a problem hiding this comment.
makes sense, can you clarify a little bit more, e.g.:
this method may degrade the original model's accuracy more compared to dynamic quantization.
we use accuracy instead of performance, since performance could refer to inference speed and accuracy
There was a problem hiding this comment.
Hi @jerryzh168, what about adding this at the end:
Converting weights and activation functions to 8-bit integers can slightly alter the network's behavior and activation responses, leading to such variations in accuracy.
There was a problem hiding this comment.
Hi @jerryzh168, I've just pushed a few changes, please let me know if you have any other suggestions. Thank you!
There was a problem hiding this comment.
@jmarintur "Converting weights and activation functions to 8-bit integers can slightly alter the network's behavior and activation responses, leading to such variations in accuracy." this is true for any quantization method, so probably better to remove I feel
Fixes #2725
Description
This PR clarifies post training quantization behaviour
Checklist