You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: prototype_source/pt2e_quant_x86_inductor.rst
+5-26Lines changed: 5 additions & 26 deletions
Original file line number
Diff line number
Diff line change
@@ -7,7 +7,7 @@ Prerequisites
7
7
^^^^^^^^^^^^^^^
8
8
9
9
- `PyTorch 2 Export Post Training Quantization <https://pytorch.org/tutorials/prototype/pt2e_quant_ptq.html>`_
10
-
- `PyTorch 2 Export Quantization-Aware Training tutorial <https://pytorch.org/tutorials/prototype/pt2e_quant_qat.html>`_
10
+
- `PyTorch 2 Export Quantization-Aware Training <https://pytorch.org/tutorials/prototype/pt2e_quant_qat.html>`_
11
11
- `TorchInductor and torch.compile concepts in PyTorch <https://pytorch.org/tutorials/intermediate/torch_compile_tutorial.html>`_
12
12
- `Inductor C++ Wrapper concepts <https://pytorch.org/tutorials/prototype/inductor_cpp_wrapper_tutorial.html>`_
13
13
@@ -17,7 +17,7 @@ Introduction
17
17
This tutorial introduces the steps for utilizing the PyTorch 2 Export Quantization flow to generate a quantized model customized
18
18
for the x86 inductor backend and explains how to lower the quantized model into the inductor.
19
19
20
-
The new quantization 2 flow uses the PT2 Export to capture the model into a graph and perform quantization transformations on top of the ATen graph.
20
+
The pytorch 2 export quantization flow uses the torch.export to capture the model into a graph and perform quantization transformations on top of the ATen graph.
21
21
This approach is expected to have significantly higher model coverage, better programmability, and a simplified UX.
22
22
TorchInductor is the new compiler backend that compiles the FX Graphs generated by TorchDynamo into optimized C++/Triton kernels.
23
23
@@ -85,8 +85,6 @@ We will start by performing the necessary imports, capturing the FX Graph from t
85
85
model = models.__dict__[model_name](pretrained=True)
86
86
87
87
# Set the model to eval mode
88
-
# Only apply it for post-training static quantization
89
-
# Skip this step for quantization-aware training
90
88
model = model.eval()
91
89
92
90
# Create the data, using the dummy data here as an example
@@ -120,44 +118,26 @@ Next, we will have the FX Module to be quantized.
120
118
After we capture the FX Module to be quantized, we will import the Backend Quantizer for X86 CPU and configure how to
The default quantization configuration in ``X86InductorQuantizer`` uses 8-bits for both activations and weights.
140
129
When Vector Neural Network Instruction is not available, the oneDNN backend silently chooses kernels that assume
141
130
`multiplications are 7-bit x 8-bit <https://oneapi-src.github.io/oneDNN/dev_guide_int8_computations.html#inputs-of-mixed-type-u8-and-s8>`_. In other words, potential
142
131
numeric saturation and accuracy issue may happen when running on CPU without Vector Neural Network Instruction.
143
132
144
-
After we import the backend-specific Quantizer, we will prepare the model for post-training quantization or quantization-aware training.
145
-
146
-
For post-training static quantization, ``prepare_pt2e`` folds BatchNorm operators into preceding Conv2d operators, and inserts observers in appropriate places in the model.
133
+
After we import the backend-specific Quantizer, we will prepare the model for post-training quantization.
134
+
``prepare_pt2e`` folds BatchNorm operators into preceding Conv2d operators, and inserts observers in appropriate places in the model.
0 commit comments