Skip to content

Commit 2b83dc2

Browse files
jerryzh168Svetlana Karslioglu
and
Svetlana Karslioglu
authored
Apply suggestions from code review
Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
1 parent b9d09f3 commit 2b83dc2

File tree

1 file changed

+9
-9
lines changed

1 file changed

+9
-9
lines changed

prototype_source/pt2e_quantizer.rst

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -304,7 +304,7 @@ functions that are used in the example:
304304

305305
A Note on IR for PT2E Quantization Flow
306306
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
307-
IR means the intermediate representation of the model, for example, ``torch`` IR (``torch.nn`` modules, ``torch.nn.functional`` ops) or ``aten`` IR (``torch.ops.aten.linear``, ...). PT2E Quantization Flow is using pre autograd aten IR (the output of `torch.export` API) so that we support training. As is shown before, we need to match the operator or operator patterns before we can do annotations on them, So the question is how do we match the pattern?
307+
**IR** stands for the **Intermediate Representation** of the model. For example, ``torch`` IR (``torch.nn`` modules, ``torch.nn.functional`` ops) or ``aten`` IR (``torch.ops.aten.linear``, ...). PT2E Quantization Flow is using pre autograd ATen IR (the output of ``torch.export`` API) so that we support training. As is shown before, we need to match the operator or operator patterns before we can do annotations on them. The question is, how do we match the pattern?
308308

309309
1. Matching ``aten`` IR directly
310310
------------------------------------
@@ -335,13 +335,13 @@ Example::
335335
# annotate conv and relu nodes
336336
...
337337

338-
However one problem for using this IR is that the representation might change if the PyTorch implementation for modules or functional ops changed. But this could be unexpected since modeling users typically assume that when the eager mode model code doesn't change, they should get the same model representation after program capture as well. One concrete effect for this problem is that if a ``Quantizer`` do annotations based on recognizing ``aten`` IR patterns, then it may fail to recognzing the pattern after PyTorch version update, and the same eager mode floating point may be left unquantized.
338+
However, an issue with this IR is that the representation might change if the PyTorch implementation for modules or functional ops changed. But this could be unexpected since modeling users typically assume that when the eager mode model code doesn't change, they should get the same model representation after program capture as well. One concrete effect for this problem is that if a ``Quantizer`` do annotations based on recognizing ``aten`` IR patterns, then it may fail to recognize the pattern after PyTorch version update, and the same eager mode floating point may be left unquantized.
339339

340340
2. Using ``SubgraphMatcher``
341341
--------------------------------
342-
Because of this, we recommend people to recognize the pattern through ``SubgraphMatcher``, through capturing a ``torch`` IR pattern (with the same program capture used for capturing the floating point model), instead of using the ``aten`` IR pattern directly.
342+
Because of this, we recommend to recognize the pattern through ``SubgraphMatcher``, through capturing a ``torch`` IR pattern (with the same program capture used for capturing the floating point model), instead of using the ``aten`` IR pattern directly.
343343

344-
Example::
344+
**Example:**::
345345

346346
def conv_relu_pattern(x, weight, bias):
347347
conv = torch.nn.functional.conv2d(x, weight, bias)
@@ -358,15 +358,15 @@ Example::
358358
inputs[1].users[0].meta["quantization_annotation"] = ...
359359
output.meta["quantization_annotation"] = ...
360360

361-
With this, the ``Quantizer`` will still be valid even when the implementation for nn modules and functionals changes, the ``aten`` IR for floating point model will change, but since we capture the pattern again instead of hardcoding the ``aten`` IR for the pattern, we'll get the updated ``aten`` IR as well and will still be able to match the pattern.
361+
With this, the ``Quantizer`` will still be valid even when the implementation for the ``nn`` modules and functionals changes, the ``aten`` IR for floating point model will change, but since we capture the pattern again instead of hardcoding the ``aten`` IR for the pattern, we'll get the updated ``aten`` IR as well and will still be able to match the pattern.
362362

363-
One caveat is that if inputs of the pattern has multiple users, we don't have a good way to identify which user node we want to annotate except for checking the aten op target.
363+
One caveat is that if inputs of the pattern has multiple users, we don't have a good way to identify which user node we want to annotate except for checking the ``aten`` op target.
364364

365-
Another caveat is that we need to make sure we have an exhaustive list of examples (e.g. 2D, 3D, 4D inputs, real v.s. symbolic inputs, training=True v.s. training=False etc.) for the pattern to make sure cover different possible ``aten`` IR outcomes captured from the ``torch`` IR pattern.
365+
Another caveat is that we need to make sure we have an exhaustive list of examples (such as 2D, 3D, 4D inputs, real vs. symbolic inputs, ``training=True`` vs. ``training=False``, and so on) for the pattern to make sure cover different possible ``aten`` IR outcomes captured from the ``torch`` IR pattern.
366366

367367
3. Using ``SubgraphMatcherWithNameNodeMap``
368368
----------------------------------------------
369-
We also introduced a different SubgraphMatcher util called ``SubgraphMatcherWithNameNodeMap`` to make it easier to query the nodes that people want to annotate.
369+
We also introduced a different ``SubgraphMatcher`` utility called ``SubgraphMatcherWithNameNodeMap`` to make it easier to query the nodes that people want to annotate.
370370

371371
Example::
372372
@@ -387,7 +387,7 @@ Example::
387387

388388
This should be easier to use than the original ``SubgraphMatcher`` around finding the node we want to annotate or use. However, it may not work if some of the operators are captured as multiple ops. For example, if ``torch.nn.functional.conv2d`` is captured as multiple ops, then the inputs ``x``, ``weight``, ``bias`` may not be the direct inputs to the final operator marked by ``conv``, in this case we'll need to revert to the same way of access mentioned in the previous example.
389389

390-
Example::
390+
**Example:**::
391391

392392
for match in matches:
393393
# find input and output of the pattern

0 commit comments

Comments
 (0)