From 6c265cb5f69941c412a7e3464861e8eee6e1ace9 Mon Sep 17 00:00:00 2001 From: Fabio Gomez Date: Wed, 31 May 2023 12:43:32 -0500 Subject: [PATCH 1/7] removed ### lines and numbered in headlines --- prototype_source/fx_graph_mode_quant_guide.rst | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index bb360861b9f..e431e4cd806 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -22,10 +22,11 @@ You can use any combination of these options: b. Write your own observed and quantized submodule -#################################################################### If the code that is not symbolically traceable does not need to be quantized, we have the following two options to run FX Graph Mode Quantization: -1.a. Symbolically trace only the code that needs to be quantized + + +a. Symbolically trace only the code that needs to be quantized ----------------------------------------------------------------- When the whole model is not symbolically traceable but the submodule we want to quantize is symbolically traceable, we can run quantization only on that submodule. @@ -69,8 +70,7 @@ Note if original model needs to be preserved, you will have to copy it yourself before calling the quantization APIs. -##################################################### -1.b. Skip symbolically trace the non-traceable code +b. Skip symbolically trace the non-traceable code --------------------------------------------------- When we have some non-traceable code in the module, and this part of code doesn’t need to be quantized, we can factor out this part of the code into a submodule and skip symbolically trace that submodule. @@ -134,8 +134,7 @@ quantization code: If the code that is not symbolically traceable needs to be quantized, we have the following two options: -########################################################## -2.a Refactor your code to make it symbolically traceable +a. Refactor your code to make it symbolically traceable -------------------------------------------------------- If it is easy to refactor the code and make the code symbolically traceable, we can refactor the code and remove the use of non-traceable constructs in python. @@ -174,8 +173,7 @@ depends on the model. -####################################################### -2.b. Write your own observed and quantized submodule +b. Write your own observed and quantized submodule ----------------------------------------------------- If the non-traceable code can’t be refactored to be symbolically traceable, From 350d1bcfb102c94ec99a31744df1360cb04503e0 Mon Sep 17 00:00:00 2001 From: Fabio Gomez Date: Wed, 31 May 2023 13:03:32 -0500 Subject: [PATCH 2/7] Fixed indentation --- prototype_source/fx_graph_mode_quant_guide.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index e431e4cd806..4b6b7859df9 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -205,8 +205,8 @@ non-traceable logic, wrapped in a module class FP32NonTraceable: ... - -2. Define observed version of FP32NonTraceable +2. Define observed version of +FP32NonTraceable .. code:: python From b499e51ead3f7f1a6a21ed8214dab5f07b948570 Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Wed, 31 May 2023 12:34:21 -0700 Subject: [PATCH 3/7] Update fx_graph_mode_quant_guide.rst --- prototype_source/fx_graph_mode_quant_guide.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index 4b6b7859df9..b57a30fae4e 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -33,6 +33,7 @@ symbolically traceable, we can run quantization only on that submodule. before: .. code:: python + class M(nn.Module): def forward(self, x): x = non_traceable_code_1(x) From 197e6b2dce32107f085be3257851e0b5ccd68e77 Mon Sep 17 00:00:00 2001 From: Fabio Gomez Date: Wed, 31 May 2023 17:29:32 -0500 Subject: [PATCH 4/7] removed numbered from titles --- prototype_source/fx_graph_mode_quant_guide.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index b57a30fae4e..322f22e3a93 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -26,7 +26,7 @@ If the code that is not symbolically traceable does not need to be quantized, we to run FX Graph Mode Quantization: -a. Symbolically trace only the code that needs to be quantized +Symbolically trace only the code that needs to be quantized ----------------------------------------------------------------- When the whole model is not symbolically traceable but the submodule we want to quantize is symbolically traceable, we can run quantization only on that submodule. @@ -71,7 +71,7 @@ Note if original model needs to be preserved, you will have to copy it yourself before calling the quantization APIs. -b. Skip symbolically trace the non-traceable code +Skip symbolically trace the non-traceable code --------------------------------------------------- When we have some non-traceable code in the module, and this part of code doesn’t need to be quantized, we can factor out this part of the code into a submodule and skip symbolically trace that submodule. @@ -135,7 +135,7 @@ quantization code: If the code that is not symbolically traceable needs to be quantized, we have the following two options: -a. Refactor your code to make it symbolically traceable +Refactor your code to make it symbolically traceable -------------------------------------------------------- If it is easy to refactor the code and make the code symbolically traceable, we can refactor the code and remove the use of non-traceable constructs in python. @@ -174,7 +174,7 @@ depends on the model. -b. Write your own observed and quantized submodule +Write your own observed and quantized submodule ----------------------------------------------------- If the non-traceable code can’t be refactored to be symbolically traceable, From 99e65c98b0823de12b5f07ab063f59085b06fdba Mon Sep 17 00:00:00 2001 From: Fabio Gomez Date: Wed, 31 May 2023 17:30:12 -0500 Subject: [PATCH 5/7] added line to show code --- prototype_source/fx_graph_mode_quant_guide.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index 322f22e3a93..bfbce25080c 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -30,6 +30,7 @@ Symbolically trace only the code that needs to be quantized ----------------------------------------------------------------- When the whole model is not symbolically traceable but the submodule we want to quantize is symbolically traceable, we can run quantization only on that submodule. + before: .. code:: python @@ -44,6 +45,7 @@ before: after: .. code:: python + class FP32Traceable(nn.Module): def forward(self, x): x = traceable_code(x) From e3f19b28135c02f62af4c60a1ff10809a1e0b757 Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Thu, 1 Jun 2023 08:30:34 -0700 Subject: [PATCH 6/7] Update fx_graph_mode_quant_guide.rst Small fix --- prototype_source/fx_graph_mode_quant_guide.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index bfbce25080c..c140952cff5 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -169,13 +169,9 @@ after: return x.permute(0, 2, 1, 3) -quantization code: - This can be combined with other approaches and the quantization code depends on the model. - - Write your own observed and quantized submodule ----------------------------------------------------- From 18ba0e07d4f1f8bdea1a13a30e1c404d62175172 Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Thu, 1 Jun 2023 08:43:01 -0700 Subject: [PATCH 7/7] Update fx_graph_mode_quant_guide.rst --- prototype_source/fx_graph_mode_quant_guide.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/prototype_source/fx_graph_mode_quant_guide.rst b/prototype_source/fx_graph_mode_quant_guide.rst index c140952cff5..9072e488a4b 100644 --- a/prototype_source/fx_graph_mode_quant_guide.rst +++ b/prototype_source/fx_graph_mode_quant_guide.rst @@ -4,7 +4,7 @@ **Author**: `Jerry Zhang `_ FX Graph Mode Quantization requires a symbolically traceable model. -We use the FX framework (TODO: link) to convert a symbolically traceable nn.Module instance to IR, +We use the FX framework to convert a symbolically traceable nn.Module instance to IR, and we operate on the IR to execute the quantization passes. Please post your question about symbolically tracing your model in `PyTorch Discussion Forum `_