@@ -22,10 +22,11 @@ You can use any combination of these options:
22
22
b. Write your own observed and quantized submodule
23
23
24
24
25
- ####################################################################
26
25
If the code that is not symbolically traceable does not need to be quantized, we have the following two options
27
26
to run FX Graph Mode Quantization:
28
- 1.a. Symbolically trace only the code that needs to be quantized
27
+
28
+
29
+ a. Symbolically trace only the code that needs to be quantized
29
30
-----------------------------------------------------------------
30
31
When the whole model is not symbolically traceable but the submodule we want to quantize is
31
32
symbolically traceable, we can run quantization only on that submodule.
@@ -69,8 +70,7 @@ Note if original model needs to be preserved, you will have to
69
70
copy it yourself before calling the quantization APIs.
70
71
71
72
72
- #####################################################
73
- 1.b. Skip symbolically trace the non-traceable code
73
+ b. Skip symbolically trace the non-traceable code
74
74
---------------------------------------------------
75
75
When we have some non-traceable code in the module, and this part of code doesn’t need to be quantized,
76
76
we can factor out this part of the code into a submodule and skip symbolically trace that submodule.
@@ -134,8 +134,7 @@ quantization code:
134
134
135
135
If the code that is not symbolically traceable needs to be quantized, we have the following two options:
136
136
137
- ##########################################################
138
- 2.a Refactor your code to make it symbolically traceable
137
+ a. Refactor your code to make it symbolically traceable
139
138
--------------------------------------------------------
140
139
If it is easy to refactor the code and make the code symbolically traceable,
141
140
we can refactor the code and remove the use of non-traceable constructs in python.
@@ -174,8 +173,7 @@ depends on the model.
174
173
175
174
176
175
177
- #######################################################
178
- 2.b. Write your own observed and quantized submodule
176
+ b. Write your own observed and quantized submodule
179
177
-----------------------------------------------------
180
178
181
179
If the non-traceable code can’t be refactored to be symbolically traceable,
0 commit comments