Adding 2 new quant tutorials #757
Conversation
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
Update Dynamic Quant for BERT tutorial
Update reference to bert_mrpc image saved in _static/img/bert_mrpc.png: https://github.com/pytorch/tutorials/blob/master/_static/img/bert_mrpc.png
Fix formatting and clean up tutorial on quantized transfer learning
Jlin tutorials quant
Minor changes in the quantized transfer learning tutorial
Update Dynamic Quant BERT Tutorial 2
Update Dynamic Quant BERT Tutorial 3
|
Deploy preview for pytorch-tutorials-preview ready! Built with commit 22f7fa4 https://deploy-preview-757--pytorch-tutorials-preview.netlify.com |
| # notice `quantize=False` | ||
| model = models.resnet18(pretrained=True, progress=True, quantize=False) | ||
| num_ftrs = model.fc.in_features | ||
|
|
||
| # Step 1 | ||
| model.train() | ||
| model.fuse_model() | ||
| # Step 2 | ||
| model_ft = create_combined_model(model) | ||
| model_ft[0].qconfig = torch.quantization.default_qat_qconfig # Use default QAT configuration | ||
| # Step 3 | ||
| model_ft = torch.quantization.prepare_qat(model_ft, inplace=True) |
There was a problem hiding this comment.
.. code:: python
# notice `quantize=False`
model = models.resnet18(pretrained=True, progress=True, quantize=False)
num_ftrs = model.fc.in_features
# Step 1
model.train()
model.fuse_model()
# Step 2
model_ft = create_combined_model(model)
model_ft[0].qconfig = torch.quantization.default_qat_qconfig # Use default QAT configuration
# Step 3
model_ft = torch.quantization.prepare_qat(model_ft, inplace=True)| the results shown in this tutorial. | ||
|
|
||
|
|
||
| # notice `quantize=False` |
There was a problem hiding this comment.
This part from 449-451 needs to be replaced with actual code:
# Step 1
model.train()
model.fuse_model()
# Step 2
model_ft = create_combined_model(model)
model_ft[0].qconfig = torch.quantization.default_qat_qconfig
# Use default QAT configuration
# Step 3
model_ft = torch.quantization.prepare_qat(model_ft, inplace=True)| inserts fake-quantization modules. | ||
|
|
||
|
|
||
| As step (4), you can start "finetuning" the model, and after that convert |
There was a problem hiding this comment.
Minor comment, suggest replacing with:
Once steps 1-3 are done, we can start fine tuning the model (step 4), followed by converting the model into a quantized. (step 5).
| python download_glue_data.py --data_dir='glue_data' --tasks='MRPC' | ||
|
|
||
|
|
||
| 1.4 Learn about helper functions |
There was a problem hiding this comment.
Suggest merging this section (1.4) with 1.2 as we are explaining what we are doing in 1.2 here, suggested change is shown above.
| logging.WARN) # Reduce logging | ||
|
|
||
| print(torch.__version__) | ||
|
|
There was a problem hiding this comment.
The helper functions are built-in in transformers library. We mainly use
the following helper functions: one for converting the text examples
into the feature vectors; The other one for measuring the F1 score of
the predicted result.
The glue_convert_examples_to_features <https://github.com/huggingface/transformers/blob/master/transformers/data/processors/glue.py>_ function converts the texts into input features:
- Tokenize the input sequences;
- Insert [CLS] in the beginning;
- Insert [SEP] between the first sentence and the second sentence, and
in the end; - Generate token type ids to indicate whether a token belongs to the
first sequence or the second sequence.
The glue_compute_metrics function computes the F1 score <https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html>_
can be interpreted as a weighted average of the precision and recall,
where an F1 score reaches its best value at 1 and worst score at 0. The
relative contribution of precision and recall to the F1 score are equal.
- The equation for the F1 score is:
.. math:: F1 = 2 * (\text{precision} * \text{recall}) / (\text{precision} + \text{recall})
Update Dynamic Quant BERT Tutorial 4
Adding 2 new quant tutorials
(experimental) Quantized Transfer Learning for Computer Vision Tutorial
https://github.com/pytorch/tutorials/blob/jlin27-quant-tutorials/intermediate_source/quantized_transfer_learning_tutorial.rst
(experimental) Dynamic Quantization on HuggingFace BERT model
https://github.com/pytorch/tutorials/blob/jlin27-quant-tutorials/intermediate_source/dynamic_quantization_bert_tutorial.rst