Update script_optimized and vulkan_workflow docs with new optimization options #2163

salilsdesai · 2023-01-02T21:15:03Z

Update to the following tutorials:

This keeps them aligned with the changes to optimize_for_mobile as of pytorch/pytorch#91600

Any changes in this PR which are deletes of space at the end of lines happened automatically when I saved the files

…n options

salilsdesai · 2023-01-02T21:17:53Z

@albanD @kimishpatel @SS-JIA Please review. I'm not able to add reviewers to this PR properly.

albanD

SGTM

recipes_source/script_optimized.rst

kimishpatel

minor change requested, but approving

svekars

A couple of super minor nits.

svekars · 2023-01-03T16:24:09Z

recipes_source/script_optimized.rst

@@ -194,16 +194,23 @@ The optimized model can then be saved and deployed in mobile apps:

    optimized_torchscript_model.save("optimized_torchscript_model.pth")

-By default, `optimize_for_mobile` will perform the following types of optimizations:
+For CPU backend, by default, `optimize_for_mobile` will perform the following types of optimizations:


Suggested change

For CPU backend, by default, `optimize_for_mobile` will perform the following types of optimizations:

By default, for the CPU backend, `optimize_for_mobile` performs the following types of optimizations:

Is there a particular reason why you wanted to move "by default" to the front? I was thinking it would be best to start with "For CPU backend" so that this section is easily distinguishable from the "For Vulkan Backend" section.

prototype_source/vulkan_workflow.rst

svekars · 2023-01-03T16:26:40Z

recipes_source/script_optimized.rst


+* `Conv packed params hoisting` which moves convolution packed params to the root module, so that the convolution structs can be deleted. This decreases model size without impacting numerics.
+
+For Vulkan backend, by default, `optimize_for_mobile` will perform the following type of optimization:


Suggested change

For Vulkan backend, by default, `optimize_for_mobile` will perform the following type of optimization:

For the Vulkan backend,`optimize_for_mobile` performs the following type of optimization:

Why was "by default" removed here but not for the CPU section above?

…pdates

Co-authored-by: Kimish Patel <kimishpatel@fb.com>

salilsdesai · 2023-01-08T01:03:01Z

prototype_source/vulkan_workflow.rst

@@ -112,6 +112,10 @@ PyTorch 1.7 Vulkan backend supports only float 32bit operators. The default mode

 The result model can be used only on Vulkan backend as it contains specific to the Vulkan backend operators.

+By default, ``optimize_for_mobile`` with ``backend='vulkan'`` rewrites the graph such that inputs are transferred to Vulkan backend, and outputs are transferred to CPU backend, so the model can be run on CPU inputs and produce CPU outputs. To disable this, add the argument ``optimization_blocklist={MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER}`` to ``optimize_for_mobile``. (``MobileOptimizerType`` can be imported from ``torch.utils.mobile_optimizer``)


I think we should change this to align with Kimish's suggestion in script_optimized.rst, i.e.

By default, ``optimize_for_mobile`` with ``backend='vulkan'`` rewrites the graph so that moving input and output data to and from the GPU becomes part of the model. To disable this, add the argument ``optimization_blocklist={MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER}`` to ``optimize_for_mobile``. (``MobileOptimizerType`` can be imported from ``torch.utils.mobile_optimizer``)

salilsdesai · 2023-01-08T01:05:44Z

recipes_source/script_optimized.rst

@@ -194,16 +194,23 @@ The optimized model can then be saved and deployed in mobile apps:

    optimized_torchscript_model.save("optimized_torchscript_model.pth")

-By default, `optimize_for_mobile` will perform the following types of optimizations:
+For CPU backend, by default, `optimize_for_mobile` will perform the following types of optimizations:


Is there a particular reason why you wanted to move "by default" to the front? I was thinking it would be best to start with "For CPU backend" so that this section is easily distinguishable from the "For Vulkan Backend" section.

salilsdesai · 2023-01-08T23:11:27Z

recipes_source/script_optimized.rst


+* `Conv packed params hoisting` which moves convolution packed params to the root module, so that the convolution structs can be deleted. This decreases model size without impacting numerics.
+


We want to add this point here:

Add/ReLU fusion which finds instances of relu ops that follow add ops and fuses them into a single add_relu.

Summary: X-link: pytorch/pytorch#91600 Pull Request resolved: facebookresearch#452 Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: kimishpatel Differential Revision: D41690203 fbshipit-source-id: 28a309c13369106181f1f22f59c6679ee6a52e6a

Summary: X-link: pytorch/pytorch#91600 Pull Request resolved: facebookresearch#452 Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: kimishpatel Differential Revision: D41690203 fbshipit-source-id: 505b507c8592deac17aee06eb320368fd4bdbb6d

Summary: X-link: pytorch/pytorch#91600 Pull Request resolved: facebookresearch#452 bypass-github-export-checks Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: kimishpatel Differential Revision: D41690203 fbshipit-source-id: 5c024d58dc6743a25832992d7a72af3d2cee1614

…on (pytorch#91600) Summary: Pull Request resolved: pytorch#91600 X-link: facebookresearch/d2go#452 bypass-github-export-checks Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Test Plan: ```buck test caffe2/test:test_mobile_optimizer``` ``` Summary Pass: 6 Skip: 1 ↻ caffe2/test:test_mobile_optimizer - test_mobilenet_optimize_for_mobile (test_mobile_optimizer.TestOptimizer) ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4222124793514412 ``` ___ ```buck test caffe2/torch/fb/mobile/tests:model_exporter_tests``` Tests pass ___ With temporary testing changes in D41690204: ```buck run caffe2:test_rewrite_flags_api``` Before: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ❌ (module 'torch._C' has no attribute '_MobileOptimizerType') torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ ``` After: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'MobileOptimizerType') torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'MobileOptimizerType') torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') ``` ```buck test caffe2/test:public_bindings -- test_no_new_bindings``` ``` Summary Pass: 1 ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/7881299473114294 ``` Reviewed By: kimishpatel Differential Revision: D41690203 fbshipit-source-id: 1ac9a8ad987cb500d38685ba9e061885eedf51e1

Summary: X-link: pytorch/pytorch#91600 Pull Request resolved: #452 bypass-github-export-checks Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: kimishpatel Differential Revision: D41690203 fbshipit-source-id: d901bdcbd16a594c3268e09b57c61b38c33a562f

…on (#91600) Summary: X-link: facebookresearch/d2go#452 Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Test Plan: ```buck test caffe2/test:test_mobile_optimizer``` ``` Summary Pass: 6 Skip: 1 ↻ caffe2/test:test_mobile_optimizer - test_mobilenet_optimize_for_mobile (test_mobile_optimizer.TestOptimizer) ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4222124793514412 ``` ___ With temporary testing changes in D41690204: ```buck run caffe2:test_rewrite_flags_api``` Before: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ❌ (module 'torch._C' has no attribute '_MobileOptimizerType') torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ ``` After: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'MobileOptimizerType') torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'MobileOptimizerType') torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') ``` ```buck test caffe2/test:public_bindings -- test_no_new_bindings``` ``` Summary Pass: 1 ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/7881299473114294 ``` Differential Revision: D41690203 Pull Request resolved: #91600 Approved by: https://github.com/albanD, https://github.com/malfet

Summary: Reland of D41690203 (facebookresearch@18de6ff) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Differential Revision: D42442395 fbshipit-source-id: 39c08d052cb023ef5870b3e0cdb7249386cfd691

Summary: Pull Request resolved: facebookresearch#459 Reland of D41690203 (facebookresearch@18de6ff) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: SS-JIA Differential Revision: D42442395 fbshipit-source-id: ababb372afce0b385e3b0a6fff430ff2bb72866c

Summary: X-link: pytorch/pytorch#92081 Pull Request resolved: facebookresearch#459 Reland of D41690203 (facebookresearch@18de6ff) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: SS-JIA Differential Revision: D42442395 fbshipit-source-id: eab0c0b9dcd83b42ccd9b01da31d595b3a453bbf

Summary: X-link: pytorch/pytorch#92081 Pull Request resolved: facebookresearch#459 Reland of D41690203 (facebookresearch@18de6ff) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: SS-JIA Differential Revision: D42442395 fbshipit-source-id: 403f70209ff1bf536a06f0e8dea1771b40270e88

Summary: X-link: pytorch/pytorch#92081 Pull Request resolved: facebookresearch#459 Reland of D41690203 (facebookresearch@18de6ff) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: SS-JIA Differential Revision: D42442395 fbshipit-source-id: 09ebc51e8c3e7116854aa347459cd80819441046

…cumentation (pytorch#92081) Summary: Pull Request resolved: pytorch#92081 X-link: facebookresearch/d2go#459 Reland of D41690203 (pytorch@370df96) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Test Plan: ```buck test caffe2/test:test_mobile_optimizer``` ``` Summary Pass: 6 Skip: 1 ↻ caffe2/test:test_mobile_optimizer - test_mobilenet_optimize_for_mobile (test_mobile_optimizer.TestOptimizer) ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4222124793514412 ``` ___ ```buck test caffe2/torch/fb/mobile/tests:model_exporter_tests``` Tests pass ___ With temporary testing changes in D41690204: ```buck run caffe2:test_rewrite_flags_api``` Before: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ❌ (module 'torch._C' has no attribute '_MobileOptimizerType') torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ ``` After: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'MobileOptimizerType') torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'MobileOptimizerType') torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') ``` ```buck test caffe2/test:public_bindings -- test_no_new_bindings``` ``` Summary Pass: 1 ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/7881299473114294 ``` Reviewed By: SS-JIA Differential Revision: D42442395 fbshipit-source-id: 65d2f6ce4b014f2cf38e9b7d24935aae43d0fc6a

Summary: X-link: pytorch/pytorch#92081 Pull Request resolved: #459 Reland of D41690203 (18de6ff) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Reviewed By: SS-JIA Differential Revision: D42442395 fbshipit-source-id: 14500b3667f541fd1ec85b1624125120176c6fd0

…cumentation (#92081) Summary: X-link: facebookresearch/d2go#459 Reland of D41690203 (370df96) Remove MobileOptimizerType and all rewrite flags from torch.X and torch._C.X to clean up torch.X and torch._C.X namespaces The affected rewrite flags are - CONV_BN_FUSION - FUSE_ADD_RELU - HOIST_CONV_PACKED_PARAMS - INSERT_FOLD_PREPACK_OPS - REMOVE_DROPOUT - VULKAN_AUTOMATIC_GPU_TRANSFER Bc-Breaking Change: Before this change, the rewrite flags were accessible through all of 1. torch.utils.mobile_optimizer.MobileOptimizerType.X 2. torch._C.MobileOptimizerType.X 3. torch.X 4. torch.MobileOptimizerType.X 5. torch._C.X But after this change, only torch.utils.mobile_optimizer.MobileOptimizerType.X (option 1 above) and the newly added torch._C._MobileOptimizerType.X remain Corresponding updates to PyTorch Tutorial Docs are in pytorch/tutorials#2163 Test Plan: ```buck test caffe2/test:test_mobile_optimizer``` ``` Summary Pass: 6 Skip: 1 ↻ caffe2/test:test_mobile_optimizer - test_mobilenet_optimize_for_mobile (test_mobile_optimizer.TestOptimizer) ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/4222124793514412 ``` ___ ```buck test caffe2/torch/fb/mobile/tests:model_exporter_tests``` Tests pass ___ With temporary testing changes in D41690204: ```buck run caffe2:test_rewrite_flags_api``` Before: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ❌ (module 'torch._C' has no attribute '_MobileOptimizerType') torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ✅ ``` After: ``` torch.utils.mobile_optimizer.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C._MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ✅ | Result: ✅ torch._C.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'MobileOptimizerType') torch.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') torch.MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch' has no attribute 'MobileOptimizerType') torch._C.VULKAN_AUTOMATIC_GPU_TRANSFER Expected: ❌ | Result: ❌ (module 'torch._C' has no attribute 'VULKAN_AUTOMATIC_GPU_TRANSFER') ``` ```buck test caffe2/test:public_bindings -- test_no_new_bindings``` ``` Summary Pass: 1 ListingSuccess: 1 Finished test run: https://www.internalfb.com/intern/testinfra/testrun/7881299473114294 ``` Reviewed By: SS-JIA Differential Revision: D42442395 Pull Request resolved: #92081 Approved by: https://github.com/albanD

Update script_optimized and vulkan_workflow docs with new optimizatio…

6474cf9

…n options

facebook-github-bot added the cla signed label Jan 2, 2023

This was referenced Jan 2, 2023

Clean Up MobileOptimizerType Rewrite Flags Public API and Documentation pytorch/pytorch#91600

Closed

Clean Up MobileOptimizerType Rewrite Flags Public API and Documentation facebookresearch/d2go#452

Closed

albanD approved these changes Jan 3, 2023

View reviewed changes

kimishpatel reviewed Jan 3, 2023

View reviewed changes

recipes_source/script_optimized.rst Outdated Show resolved Hide resolved

kimishpatel approved these changes Jan 3, 2023

View reviewed changes

svekars reviewed Jan 3, 2023

View reviewed changes

malfet approved these changes Jan 4, 2023

View reviewed changes

Svetlana Karslioglu and others added 5 commits January 4, 2023 08:16

Merge branch 'main' into optimize_for_mobile_rewrite_flags_tutorial_u…

c98d34c

…pdates

grammar

e902297

grammar

bd731cf

grammar

866262e

grammar

c1ff94a

Co-authored-by: Kimish Patel <kimishpatel@fb.com>

svekars merged commit a0d4ce6 into pytorch:main Jan 4, 2023

salilsdesai commented Jan 8, 2023

View reviewed changes

salilsdesai mentioned this pull request Jan 10, 2023

[Reland] Clean Up MobileOptimizerType Rewrite Flags Public API and Documentation facebookresearch/d2go#459

Closed

salilsdesai mentioned this pull request Jan 12, 2023

[Reland] Clean Up MobileOptimizerType Rewrite Flags Public API and Documentation pytorch/pytorch#92081

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update script_optimized and vulkan_workflow docs with new optimization options #2163

Update script_optimized and vulkan_workflow docs with new optimization options #2163

Uh oh!

salilsdesai commented Jan 2, 2023

Uh oh!

salilsdesai commented Jan 2, 2023

Uh oh!

albanD left a comment

Uh oh!

Uh oh!

kimishpatel left a comment

Uh oh!

svekars left a comment

Uh oh!

svekars Jan 3, 2023

Uh oh!

salilsdesai Jan 8, 2023

Uh oh!

Uh oh!

svekars Jan 3, 2023

Uh oh!

salilsdesai Jan 8, 2023

Uh oh!

salilsdesai Jan 8, 2023

Uh oh!

salilsdesai Jan 8, 2023

Uh oh!

salilsdesai Jan 8, 2023

Uh oh!

Uh oh!

	For CPU backend, by default, `optimize_for_mobile` will perform the following types of optimizations:
	By default, for the CPU backend, `optimize_for_mobile` performs the following types of optimizations:


		* `Conv packed params hoisting` which moves convolution packed params to the root module, so that the convolution structs can be deleted. This decreases model size without impacting numerics.

		For Vulkan backend, by default, `optimize_for_mobile` will perform the following type of optimization:

	For Vulkan backend, by default, `optimize_for_mobile` will perform the following type of optimization:
	For the Vulkan backend,`optimize_for_mobile` performs the following type of optimization:

		@@ -112,6 +112,10 @@ PyTorch 1.7 Vulkan backend supports only float 32bit operators. The default mode

		The result model can be used only on Vulkan backend as it contains specific to the Vulkan backend operators.

		By default, ``optimize_for_mobile`` with ``backend='vulkan'`` rewrites the graph such that inputs are transferred to Vulkan backend, and outputs are transferred to CPU backend, so the model can be run on CPU inputs and produce CPU outputs. To disable this, add the argument ``optimization_blocklist={MobileOptimizerType.VULKAN_AUTOMATIC_GPU_TRANSFER}`` to ``optimize_for_mobile``. (``MobileOptimizerType`` can be imported from ``torch.utils.mobile_optimizer``)

Update script_optimized and vulkan_workflow docs with new optimization options #2163

Update script_optimized and vulkan_workflow docs with new optimization options #2163

Uh oh!

Conversation

salilsdesai commented Jan 2, 2023

Uh oh!

salilsdesai commented Jan 2, 2023

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kimishpatel left a comment

Choose a reason for hiding this comment

Uh oh!

svekars left a comment

Choose a reason for hiding this comment

Uh oh!

svekars Jan 3, 2023

Choose a reason for hiding this comment

Uh oh!

salilsdesai Jan 8, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

svekars Jan 3, 2023

Choose a reason for hiding this comment

Uh oh!

salilsdesai Jan 8, 2023

Choose a reason for hiding this comment

Uh oh!

salilsdesai Jan 8, 2023

Choose a reason for hiding this comment

Uh oh!

salilsdesai Jan 8, 2023

Choose a reason for hiding this comment

Uh oh!

salilsdesai Jan 8, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!