diff --git a/beginner_source/deeplabv3_on_android.rst b/beginner_source/deeplabv3_on_android.rst index 5ca7f01ad06..7ec83477373 100644 --- a/beginner_source/deeplabv3_on_android.rst +++ b/beginner_source/deeplabv3_on_android.rst @@ -1,234 +1,10 @@ Image Segmentation DeepLabV3 on Android ================================================= -**Author**: `Jeff Tang `_ +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -**Reviewed by**: `Jeremiah Chung `_ +Redirecting in 3 seconds... -.. warning:: - PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `_, PyTorch’s all-new on-device inference library. You can also review our `end-to-end workflows `_ and review the `source code for DeepLabV3 `_. +.. raw:: html - -Introduction ------------- - -Semantic image segmentation is a computer vision task that uses semantic labels to mark specific regions of an input image. The PyTorch semantic image segmentation `DeepLabV3 model `_ can be used to label image regions with `20 semantic classes `_ including, for example, bicycle, bus, car, dog, and person. Image segmentation models can be very useful in applications such as autonomous driving and scene understanding. - -In this tutorial, we will provide a step-by-step guide on how to prepare and run the PyTorch DeepLabV3 model on Android, taking you from the beginning of having a model you may want to use on Android to the end of having a complete Android app using the model. We will also cover practical and general tips on how to check if your next favorable pretrained PyTorch models can run on Android, and how to avoid pitfalls. - -.. note:: Before going through this tutorial, you should check out `PyTorch Mobile for Android `_ and give the PyTorch Android `Hello World `_ example app a quick try. This tutorial will go beyond the image classification model, usually the first kind of model deployed on mobile. The complete code for this tutorial is available `here `_. - -Learning Objectives -------------------- - -In this tutorial, you will learn how to: - -1. Convert the DeepLabV3 model for Android deployment. - -2. Get the output of the model for the example input image in Python and compare it to the output from the Android app. - -3. Build a new Android app or reuse an Android example app to load the converted model. - -4. Prepare the input into the format that the model expects and process the model output. - -5. Complete the UI, refactor, build and run the app to see image segmentation in action. - -Prerequisites ---------------- - -* PyTorch 1.6 or 1.7 - -* torchvision 0.7 or 0.8 - -* Android Studio 3.5.1 or above with NDK installed - -Steps ---------- - -1. Convert the DeepLabV3 model for Android deployment -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The first step to deploying a model on Android is to convert the model into the `TorchScript `_ format. - -.. note:: - Not all PyTorch models can be converted to TorchScript at this time because a model definition may use language features that are not in TorchScript, which is a subset of Python. See the `Script and Optimize Recipe <../recipes/script_optimized.html>`_ for more details. - -Simply run the script below to generate the scripted model `deeplabv3_scripted.pt`: - -:: - - import torch - - # use deeplabv3_resnet50 instead of resnet101 to reduce the model size - model = torch.hub.load('pytorch/vision:v0.7.0', 'deeplabv3_resnet50', pretrained=True) - model.eval() - - scriptedm = torch.jit.script(model) - torch.jit.save(scriptedm, "deeplabv3_scripted.pt") - -The size of the generated `deeplabv3_scripted.pt` model file should be around 168MB. Ideally, a model should also be quantized for significant size reduction and faster inference before being deployed on an Android app. To have a general understanding of quantization, see the `Quantization Recipe <../recipes/quantization.html>`_ and the resource links there. We will cover in detail how to correctly apply a quantization workflow called Post Training `Static Quantization `_ to the DeepLabV3 model in a future tutorial or recipe. - -2. Get example input and output of the model in Python -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Now that we have a scripted PyTorch model, let's test with some example inputs to make sure the model works correctly on Android. First, let's write a Python script that uses the model to make inferences and examine inputs and outputs. For this example of the DeepLabV3 model, we can reuse the code in Step 1 and in the `DeepLabV3 model hub site `_. Add the following code snippet to the code above: - -:: - - from PIL import Image - from torchvision import transforms - input_image = Image.open("deeplab.jpg") - preprocess = transforms.Compose([ - transforms.ToTensor(), - transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), - ]) - - input_tensor = preprocess(input_image) - input_batch = input_tensor.unsqueeze(0) - with torch.no_grad(): - output = model(input_batch)['out'][0] - - print(input_batch.shape) - print(output.shape) - -Download `deeplab.jpg` from `here `_, then run the script above and you will see the shapes of the input and output of the model: - -:: - - torch.Size([1, 3, 400, 400]) - torch.Size([21, 400, 400]) - -So if you provide the same image input `deeplab.jpg` of size 400x400 to the model on Android, the output of the model should have the size [21, 400, 400]. You should also print out at least the beginning parts of the actual data of the input and output, to be used in Step 4 below to compare with the actual input and output of the model when running in the Android app. - -3. Build a new Android app or reuse an example app and load the model -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -First, follow Step 3 of the `Model Preparation for Android recipe <../recipes/model_preparation_android.html#add-the-model-and-pytorch-library-on-android>`_ to use our model in an Android Studio project with PyTorch Mobile enabled. Because both DeepLabV3 used in this tutorial and MobileNet v2 used in the PyTorch Hello World Android example are computer vision models, you can also get the `Hello World example repo `_ to make it easier to modify the code that loads the model and processes the input and output. The main goal in this step and Step 4 is to make sure the model `deeplabv3_scripted.pt` generated in Step 1 can indeed work correctly on Android. - -Now let's add `deeplabv3_scripted.pt` and `deeplab.jpg` used in Step 2 to the Android Studio project and modify the `onCreate` method in the `MainActivity` to resemble: - -.. code-block:: java - - Module module = null; - try { - module = Module.load(assetFilePath(this, "deeplabv3_scripted.pt")); - } catch (IOException e) { - Log.e("ImageSegmentation", "Error loading model!", e); - finish(); - } - -Then set a breakpoint at the line `finish()` and build and run the app. If the app doesn't stop at the breakpoint, it means that the scripted model in Step 1 has been successfully loaded on Android. - -4. Process the model input and output for model inference -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -After the model loads in the previous step, let's verify that it works with expected inputs and can generate expected outputs. As the model input for the DeepLabV3 model is an image the same as that of the MobileNet v2 in the Hello World example, we will reuse some of the code in the `MainActivity.java `_ file from Hello World for input processing. Replace the code snippet between `line 50 `_ and 73 in `MainActivity.java` with the following code: - -.. code-block:: java - - final Tensor inputTensor = TensorImageUtils.bitmapToFloat32Tensor(bitmap, - TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, - TensorImageUtils.TORCHVISION_NORM_STD_RGB); - final float[] inputs = inputTensor.getDataAsFloatArray(); - - Map outTensors = - module.forward(IValue.from(inputTensor)).toDictStringKey(); - - // the key "out" of the output tensor contains the semantic masks - // see https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101 - final Tensor outputTensor = outTensors.get("out").toTensor(); - final float[] outputs = outputTensor.getDataAsFloatArray(); - - int width = bitmap.getWidth(); - int height = bitmap.getHeight(); - -.. note:: - The model output is a dictionary for the DeepLabV3 model so we use `toDictStringKey` to correctly extract the result. For other models, the model output may also be a single tensor or a tuple of tensors, among other things. - -With the code changes shown above, you can set breakpoints after `final float[] inputs` and `final float[] outputs`, which populate the input tensor and output tensor data to float arrays for easy debugging. Run the app and when it stops at the breakpoints, compare the numbers in `inputs` and `outputs` with the model input and output data you see in Step 2 to see if they match. For the same inputs to the models running on Android and Python, you should get the same outputs. - -.. warning:: - You may see different model outputs with the same image input when running on an Android emulator due to some Android emulator's floating point implementation issue. So it is best to test the app on a real Android device. - -All we have done so far is to confirm that the model of our interest can be scripted and run correctly in our Android app as in Python. The steps we walked through so far for using a model in an iOS app consumes the bulk, if not most, of our app development time, similar to how data preprocessing is the heaviest lift for a typical machine learning project. - -5. Complete the UI, refactor, build and run the app -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Now we are ready to complete the app and the UI to actually see the processed result as a new image. The output processing code should be like this, added to the end of the code snippet in Step 4: - -.. code-block:: java - - int[] intValues = new int[width * height]; - // go through each element in the output of size [WIDTH, HEIGHT] and - // set different color for different classnum - for (int j = 0; j < width; j++) { - for (int k = 0; k < height; k++) { - // maxi: the index of the 21 CLASSNUM with the max probability - int maxi = 0, maxj = 0, maxk = 0; - double maxnum = -100000.0; - for (int i=0; i < CLASSNUM; i++) { - if (outputs[i*(width*height) + j*width + k] > maxnum) { - maxnum = outputs[i*(width*height) + j*width + k]; - maxi = i; maxj = j; maxk= k; - } - } - // color coding for person (red), dog (green), sheep (blue) - // black color for background and other classes - if (maxi == PERSON) - intValues[maxj*width + maxk] = 0xFFFF0000; // red - else if (maxi == DOG) - intValues[maxj*width + maxk] = 0xFF00FF00; // green - else if (maxi == SHEEP) - intValues[maxj*width + maxk] = 0xFF0000FF; // blue - else - intValues[maxj*width + maxk] = 0xFF000000; // black - } - } - -The constants used in the code above are defined in the beginning of the class `MainActivity`: - -.. code-block:: java - - private static final int CLASSNUM = 21; - private static final int DOG = 12; - private static final int PERSON = 15; - private static final int SHEEP = 17; - - -The implementation here is based on the understanding of the DeepLabV3 model which outputs a tensor of size [21, width, height] for an input image of width*height. Each element in the width*height output array is a value between 0 and 20 (for a total of 21 semantic labels described in Introduction) and the value is used to set a specific color. Color coding of the segmentation here is based on the class with the highest probability, and you can extend the color coding for all classes in your own dataset. - -After the output processing, you will also need to call the code below to render the RGB `intValues` array to a bitmap instance `outputBitmap` before displaying it on an `ImageView`: - -.. code-block:: java - - Bitmap bmpSegmentation = Bitmap.createScaledBitmap(bitmap, width, height, true); - Bitmap outputBitmap = bmpSegmentation.copy(bmpSegmentation.getConfig(), true); - outputBitmap.setPixels(intValues, 0, outputBitmap.getWidth(), 0, 0, - outputBitmap.getWidth(), outputBitmap.getHeight()); - imageView.setImageBitmap(outputBitmap); - -The UI for this app is also similar to that for Hello World, except that you do not need the `TextView` to show the image classification result. You can also add two buttons `Segment` and `Restart` as shown in the code repository to run the model inference and to show back the original image after the segmentation result is shown. - -Now when you run the app on an Android emulator or preferably an actual device, you will see screens like the following: - -.. image:: /_static/img/deeplabv3_android.png - :width: 300 px -.. image:: /_static/img/deeplabv3_android2.png - :width: 300 px - - -Recap --------- - -In this tutorial, we described what it takes to convert a pretrained PyTorch DeepLabV3 model for Android and how to make sure the model can run successfully on Android. Our focus was to help you understand the process of confirming that a model can indeed run on Android. The complete code repository is available `here `_. - -More advanced topics such as quantization and using models via transfer learning or of your own on Android will be covered soon in future demo apps and tutorials. - - -Learn More ------------- - -1. `PyTorch Mobile site `_ -2. `DeepLabV3 model `_ -3. `DeepLabV3 paper `_ + diff --git a/beginner_source/deeplabv3_on_ios.rst b/beginner_source/deeplabv3_on_ios.rst index 5a88c703bd8..66c052419fc 100644 --- a/beginner_source/deeplabv3_on_ios.rst +++ b/beginner_source/deeplabv3_on_ios.rst @@ -1,248 +1,10 @@ Image Segmentation DeepLabV3 on iOS ============================================== -**Author**: `Jeff Tang `_ +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -**Reviewed by**: `Jeremiah Chung `_ +Redirecting in 3 seconds... -Introduction ------------- +.. raw:: html -Semantic image segmentation is a computer vision task that uses semantic labels to mark specific regions of an input image. The PyTorch semantic image segmentation `DeepLabV3 model `_ can be used to label image regions with `20 semantic classes `_ including, for example, bicycle, bus, car, dog, and person. Image segmentation models can be very useful in applications such as autonomous driving and scene understanding. - -In this tutorial, we will provide a step-by-step guide on how to prepare and run the PyTorch DeepLabV3 model on iOS, taking you from the beginning of having a model you may want to use on iOS to the end of having a complete iOS app using the model. We will also cover practical and general tips on how to check if your next favorite pretrained PyTorch models can run on iOS, and how to avoid pitfalls. - -.. note:: Before going through this tutorial, you should check out `PyTorch Mobile for iOS `_ and give the PyTorch iOS `HelloWorld `_ example app a quick try. This tutorial will go beyond the image classification model, usually the first kind of model deployed on mobile. The complete code for this tutorial is available `here `_. - -Learning Objectives -------------------- - -In this tutorial, you will learn how to: - -1. Convert the DeepLabV3 model for iOS deployment. - -2. Get the output of the model for the example input image in Python and compare it to the output from the iOS app. - -3. Build a new iOS app or reuse an iOS example app to load the converted model. - -4. Prepare the input into the format that the model expects and process the model output. - -5. Complete the UI, refactor, build and run the app to see image segmentation in action. - -Prerequisites ---------------- - -* PyTorch 1.6 or 1.7 - -* torchvision 0.7 or 0.8 - -* Xcode 11 or 12 - -Steps ---------- - - -1. Convert the DeepLabV3 model for iOS deployment -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The first step to deploying a model on iOS is to convert the model into the `TorchScript `_ format. - -.. note:: - Not all PyTorch models can be converted to TorchScript at this time because a model definition may use language features that are not in TorchScript, which is a subset of Python. See the `Script and Optimize Recipe <../recipes/script_optimized.html>`_ for more details. - -Simply run the script below to generate the scripted model `deeplabv3_scripted.pt`: - -:: - - import torch - - # use deeplabv3_resnet50 instead of deeplabv3_resnet101 to reduce the model size - model = torch.hub.load('pytorch/vision:v0.8.0', 'deeplabv3_resnet50', pretrained=True) - model.eval() - - scriptedm = torch.jit.script(model) - torch.jit.save(scriptedm, "deeplabv3_scripted.pt") - -The size of the generated `deeplabv3_scripted.pt` model file should be around 168MB. Ideally, a model should also be quantized for significant size reduction and faster inference before being deployed on an iOS app. To have a general understanding of quantization, see the `Quantization Recipe <../recipes/quantization.html>`_ and the resource links there. We will cover in detail how to correctly apply a quantization workflow called Post Training `Static Quantization `_ to the DeepLabV3 model in a future tutorial or recipe. - -2. Get example input and output of the model in Python -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Now that we have a scripted PyTorch model, let's test with some example inputs to make sure the model works correctly on iOS. First, let's write a Python script that uses the model to make inferences and examine inputs and outputs. For this example of the DeepLabV3 model, we can reuse the code in Step 1 and in the `DeepLabV3 model hub site `_. Add the following code snippet to the code above: - -:: - - from PIL import Image - from torchvision import transforms - input_image = Image.open("deeplab.jpg") - preprocess = transforms.Compose([ - transforms.ToTensor(), - transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), - ]) - - input_tensor = preprocess(input_image) - input_batch = input_tensor.unsqueeze(0) - with torch.no_grad(): - output = model(input_batch)['out'][0] - - print(input_batch.shape) - print(output.shape) - -Download `deeplab.jpg` from `here `_ and run the script above to see the shapes of the input and output of the model: - -:: - - torch.Size([1, 3, 400, 400]) - torch.Size([21, 400, 400]) - -So if you provide the same image input `deeplab.jpg` of size 400x400 to the model on iOS, the output of the model should have the size [21, 400, 400]. You should also print out at least the beginning parts of the actual data of the input and output, to be used in Step 4 below to compare with the actual input and output of the model when running in the iOS app. - -3. Build a new iOS app or reuse an example app and load the model -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -First, follow Step 3 of the `Model Preparation for iOS recipe <../recipes/model_preparation_ios.html#add-the-model-and-pytorch-library-on-ios>`_ to use our model in an Xcode project with PyTorch Mobile enabled. Because both the DeepLabV3 model used in this tutorial and the MobileNet v2 model used in the PyTorch Hello World iOS example are computer vision models, you may choose to start with the `HelloWorld example repo `_ as a template to reuse the code that loads the model and processes the input and output. - -Now let's add `deeplabv3_scripted.pt` and `deeplab.jpg` used in Step 2 to the Xcode project and modify `ViewController.swift` to resemble: - -.. code-block:: swift - - class ViewController: UIViewController { - var image = UIImage(named: "deeplab.jpg")! - - override func viewDidLoad() { - super.viewDidLoad() - } - - private lazy var module: TorchModule = { - if let filePath = Bundle.main.path(forResource: "deeplabv3_scripted", - ofType: "pt"), - let module = TorchModule(fileAtPath: filePath) { - return module - } else { - fatalError("Can't load the model file!") - } - }() - } - -Then set a breakpoint at the line `return module` and build and run the app. The app should stop at the breakpoint, meaning that the scripted model in Step 1 has been successfully loaded on iOS. - -4. Process the model input and output for model inference -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -After the model loads in the previous step, let's verify that it works with expected inputs and can generate expected outputs. As the model input for the DeepLabV3 model is an image, the same as that of the MobileNet v2 in the Hello World example, we will reuse some of the code in the `TorchModule.mm `_ file from Hello World for input processing. Replace the `predictImage` method implementation in `TorchModule.mm` with the following code: - -.. code-block:: objective-c - - - (unsigned char*)predictImage:(void*)imageBuffer { - // 1. the example deeplab.jpg size is size 400x400 and there are 21 semantic classes - const int WIDTH = 400; - const int HEIGHT = 400; - const int CLASSNUM = 21; - - at::Tensor tensor = torch::from_blob(imageBuffer, {1, 3, WIDTH, HEIGHT}, at::kFloat); - torch::autograd::AutoGradMode guard(false); - at::AutoNonVariableTypeMode non_var_type_mode(true); - - // 2. convert the input tensor to an NSMutableArray for debugging - float* floatInput = tensor.data_ptr(); - if (!floatInput) { - return nil; - } - NSMutableArray* inputs = [[NSMutableArray alloc] init]; - for (int i = 0; i < 3 * WIDTH * HEIGHT; i++) { - [inputs addObject:@(floatInput[i])]; - } - - // 3. the output of the model is a dictionary of string and tensor, as - // specified at https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101 - auto outputDict = _impl.forward({tensor}).toGenericDict(); - - // 4. convert the output to another NSMutableArray for easy debugging - auto outputTensor = outputDict.at("out").toTensor(); - float* floatBuffer = outputTensor.data_ptr(); - if (!floatBuffer) { - return nil; - } - NSMutableArray* results = [[NSMutableArray alloc] init]; - for (int i = 0; i < CLASSNUM * WIDTH * HEIGHT; i++) { - [results addObject:@(floatBuffer[i])]; - } - - return nil; - } - -.. note:: - The model output is a dictionary for the DeepLabV3 model so we use `toGenericDict` to correctly extract the result. For other models, the model output may also be a single tensor or a tuple of tensors, among other things. - -With the code changes shown above, you can set breakpoints after the two for loops that populate `inputs` and `results` and compare them with the model input and output data you saw in Step 2 to see if they match. For the same inputs to the models running on iOS and Python, you should get the same outputs. - -All we have done so far is to confirm that the model of our interest can be scripted and run correctly in our iOS app as in Python. The steps we walked through so far for using a model in an iOS app consumes the bulk, if not most, of our app development time, similar to how data preprocessing is the heaviest lift for a typical machine learning project. - -5. Complete the UI, refactor, build and run the app -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Now we are ready to complete the app and the UI to actually see the processed result as a new image. The output processing code should be like this, added to the end of the code snippet in Step 4 in `TorchModule.mm` - remember to first remove the line `return nil;` temporarily put there to make the code build and run: - -.. code-block:: objective-c - - // see the 20 semantic classes link in Introduction - const int DOG = 12; - const int PERSON = 15; - const int SHEEP = 17; - - NSMutableData* data = [NSMutableData dataWithLength: - sizeof(unsigned char) * 3 * WIDTH * HEIGHT]; - unsigned char* buffer = (unsigned char*)[data mutableBytes]; - // go through each element in the output of size [WIDTH, HEIGHT] and - // set different color for different classnum - for (int j = 0; j < WIDTH; j++) { - for (int k = 0; k < HEIGHT; k++) { - // maxi: the index of the 21 CLASSNUM with the max probability - int maxi = 0, maxj = 0, maxk = 0; - float maxnum = -100000.0; - for (int i = 0; i < CLASSNUM; i++) { - if ([results[i * (WIDTH * HEIGHT) + j * WIDTH + k] floatValue] > maxnum) { - maxnum = [results[i * (WIDTH * HEIGHT) + j * WIDTH + k] floatValue]; - maxi = i; maxj = j; maxk = k; - } - } - int n = 3 * (maxj * width + maxk); - // color coding for person (red), dog (green), sheep (blue) - // black color for background and other classes - buffer[n] = 0; buffer[n+1] = 0; buffer[n+2] = 0; - if (maxi == PERSON) buffer[n] = 255; - else if (maxi == DOG) buffer[n+1] = 255; - else if (maxi == SHEEP) buffer[n+2] = 255; - } - } - return buffer; - -The implementation here is based on the understanding of the DeepLabV3 model which outputs a tensor of size [21, width, height] for an input image of width*height. Each element in the width*height output array is a value between 0 and 20 (for a total of 21 semantic labels described in Introduction) and the value is used to set a specific color. Color coding of the segmentation here is based on the class with the highest probability, and you can extend the color coding for all classes in your own dataset. - -After the output processing, you will also need to call a helper function to convert the RGB `buffer` to an `UIImage` instance to be shown on `UIImageView`. You can refer to the example code `convertRGBBufferToUIImage` defined in `UIImageHelper.mm` in the code repository. - -The UI for this app is also similar to that for Hello World, except that you do not need the `UITextView` to show the image classification result. You can also add two buttons `Segment` and `Restart` as shown in the code repository to run the model inference and to show back the original image after the segmentation result is shown. - -The last step before we can run the app is to connect all the pieces together. Modify the `ViewController.swift` file to use the `predictImage`, which is refactored and changed to `segmentImage` in the repository, and helper functions you built as shown in the example code in the repository in `ViewController.swift`. Connect the buttons to the actions and you should be good to go. - -Now when you run the app on an iOS simulator or an actual iOS device, you will see the following screens: - -.. image:: /_static/img/deeplabv3_ios.png - :width: 300 px -.. image:: /_static/img/deeplabv3_ios2.png - :width: 300 px - - -Recap --------- - -In this tutorial, we described what it takes to convert a pretrained PyTorch DeepLabV3 model for iOS and how to make sure the model can run successfully on iOS. Our focus was to help you understand the process of confirming that a model can indeed run on iOS. The complete code repository is available `here `_. - -More advanced topics such as quantization and using models via transfer learning or of your own on iOS will be covered soon in future demo apps and tutorials. - -Learn More ------------- - -1. `PyTorch Mobile site `_ -2. `DeepLabV3 model `_ -3. `DeepLabV3 paper `_ + diff --git a/prototype_source/ios_coreml_workflow.rst b/prototype_source/ios_coreml_workflow.rst index bfaccd77a10..db9abcc5076 100644 --- a/prototype_source/ios_coreml_workflow.rst +++ b/prototype_source/ios_coreml_workflow.rst @@ -1,128 +1,10 @@ (Prototype) Convert Mobilenetv2 to Core ML ========================================== -**Author**: `Tao Xu `_ +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -Introduction ------------- +Redirecting in 3 seconds... -Core ML provides access to powerful and efficient NPUs(Neural Process Unit) on modern iPhone devices. This tutorial shows how to prepare a computer vision model (mobilenetv2) to use the PyTorch Core ML mobile backend. +.. raw:: html -Note that this feature is currently in the “prototype” phase and only supports a limited numbers of operators, but we expect to solidify the integration and expand our operator support over time. The APIs are subject to change in the future. - -Environment Setup (MacOS) -------------------------- - -Let's start off by creating a new conda environment. - -.. code:: shell - - conda create --name 1.10 python=3.8 --yes - conda activate 1.10 - -Next, since the Core ML delegate is a prototype feature, let's install the PyTorch nightly build and coremltools - -.. code:: shell - - pip3 install --pre torch torchvision torchaudio -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html - - pip3 install coremltools==5.0b5 protobuf==3.20.1 - - -Model Preparation -------------------- - -To convert a pre-trained mobilenetv2 model to be Core ML compatible, we're going to use the ``to_backend()`` API, which is a prototype feature for delegating model executions to some specific backends. The following python code shows how to use it to convert the mobilenetv2 torchscript model. - -.. code:: python - - import torch - import torchvision - - from torch.backends._coreml.preprocess import ( - CompileSpec, - TensorSpec, - CoreMLComputeUnit, - ) - - def mobilenetv2_spec(): - return { - "forward": CompileSpec( - inputs=( - TensorSpec( - shape=[1, 3, 224, 224], - ), - ), - outputs=( - TensorSpec( - shape=[1, 1000], - ), - ), - backend=CoreMLComputeUnit.ALL, - allow_low_precision=True, - ), - } - - - def main(): - model = torchvision.models.mobilenet_v2(pretrained=True) - model.eval() - example = torch.rand(1, 3, 224, 224) - model = torch.jit.trace(model, example) - compile_spec = mobilenetv2_spec() - mlmodel = torch._C._jit_to_backend("coreml", model, compile_spec) - mlmodel._save_for_lite_interpreter("./mobilenetv2_coreml.ptl") - - - if __name__ == "__main__": - main() - - -First, we need to call ``.eval()`` to set the model to inference mode. Secondly, we defined a ``mobilenetv2_spec()`` function to tell Core ML what the model looks like. Note that the ``CoreMLComputeUnit`` corresponds to `Apple's processing unit `_ whose value can be ``CPU``, ``CPUAndGPU`` and ``ALL``. In our example, we set the ``backend`` type to ``ALL`` which means Core ML will try to run the model on Neural Engine. Finally, we called the ``to_backend`` API to convert the torchscript model to a Core ML compatible model and save it to the disk. - -Run the python script. If everything works well, you should see following outputs from coremltools - -.. code:: shell - - Converting Frontend ==> MIL Ops: 100%|███████████████████████████████████████████████████████████████████████████████▊| 384/385 [00:00<00:00, 1496.98 ops/s] - Running MIL Common passes: 0%| - 0/33 [00:00 NeuralNetwork Ops: 100%|██████████████████████████████████████████████████████████████████████████| 495/495 [00:00<00:00, 1977.15 ops/s] - [W backend_detail.cpp:376] Warning: Backend [coreml] is not available. Execution of this Module is still possible by saving and loading on a device where the backend is available. (function codegen_backend_module) - -We can safely ignore the warning above, as we don't plan to run our model on desktop. - -iOS app integration ---------------------- - -Now that the model is ready, we can integrate it to our app. We'll be using the pytorch nightly cocoapods which contains the code for executing the Core ML model. Simply add the following code to your Podfile - -.. code:: shell - - pod LibTorch-Lite-Nightly - -In this tutorial, we'll be reusing our `HelloWorld `_ project. Feel free to walk through the code there. - -To benchmark the latency, you can simply put the following code before and after the PyTorch ``forward`` function - -.. code:: objective-c - - caffe2::Timer t; - auto outputTensor = _impl.forward({tensor}).toTensor().cpu(); - std::cout << "forward took: " << t.MilliSeconds() << std::endl; - -Conclusion ----------- - -In this tutorial, we demonstrated how to convert a mobilenetv2 model to a Core ML compatible model. Please be aware of that Core ML feature is still under development, new operators/models will continue to be added. APIs are subject to change in the future versions. - -Thanks for reading! As always, we welcome any feedback, so please create an issue `here `_ if you have any. - -Learn More ----------- - -- The `Mobilenetv2 `_ from Torchvision -- Information about `Core ML `_ + diff --git a/prototype_source/ios_gpu_workflow.rst b/prototype_source/ios_gpu_workflow.rst index cb7a0034b23..8915e1c4fad 100644 --- a/prototype_source/ios_gpu_workflow.rst +++ b/prototype_source/ios_gpu_workflow.rst @@ -1,142 +1,10 @@ (Prototype) Use iOS GPU in PyTorch ================================== -**Author**: `Tao Xu `_ +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -Introduction ------------- +Redirecting in 3 seconds... -This tutorial introduces the steps to run your models on iOS GPU. We'll be using the mobilenetv2 model as an example. Since the mobile GPU features are currently in the prototype stage, you'll need to build a custom pytorch binary from source. For the time being, only a limited number of operators are supported, and certain client side APIs are subject to change in the future versions. +.. raw:: html -Model Preparation -------------------- - -Since GPUs consume weights in a different order, the first step we need to do is to convert our TorchScript model to a GPU compatible model. This step is also known as "prepacking". - -PyTorch with Metal -^^^^^^^^^^^^^^^^^^ -To do that, we'll install a pytorch nightly binary that includes the Metal backend. Go ahead run the command below - -.. code:: shell - - conda install pytorch -c pytorch-nightly - // or - pip3 install --pre torch -f https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html - -Also, you can build a custom pytorch binary from source that includes the Metal backend. Just checkout the pytorch source code from github and run the command below - -.. code:: shell - - cd PYTORCH_ROOT - USE_PYTORCH_METAL_EXPORT=ON python setup.py install --cmake - -The command above will build a custom pytorch binary from master. The ``install`` argument simply tells ``setup.py`` to override the existing PyTorch on your desktop. Once the build finished, open another terminal to check the PyTorch version to see if the installation was successful. As the time of writing of this recipe, the version is ``1.8.0a0+41237a4``. You might be seeing different numbers depending on when you check out the code from master, but it should be greater than 1.7.0. - -.. code:: python - - import torch - torch.__version__ #1.8.0a0+41237a4 - -Metal Compatible Model -^^^^^^^^^^^^^^^^^^^^^^ - -The next step is going to be converting the mobilenetv2 torchscript model to a Metal compatible model. We'll be leveraging the ``optimize_for_mobile`` API from the ``torch.utils`` module. As shown below - -.. code:: python - - import torch - import torchvision - from torch.utils.mobile_optimizer import optimize_for_mobile - - model = torchvision.models.mobilenet_v2(pretrained=True) - scripted_model = torch.jit.script(model) - optimized_model = optimize_for_mobile(scripted_model, backend='metal') - print(torch.jit.export_opnames(optimized_model)) - optimized_model._save_for_lite_interpreter('./mobilenetv2_metal.pt') - -Note that the ``torch.jit.export_opnames(optimized_model)`` is going to dump all the optimized operators from the ``optimized_mobile``. If everything works well, you should be able to see the following ops being printed out from the console - - -.. code:: shell - - ['aten::adaptive_avg_pool2d', - 'aten::add.Tensor', - 'aten::addmm', - 'aten::reshape', - 'aten::size.int', - 'metal::copy_to_host', - 'metal_prepack::conv2d_run'] - -Those are all the ops we need to run the mobilenetv2 model on iOS GPU. Cool! Now that you have the ``mobilenetv2_metal.pt`` saved on your disk, let's move on to the iOS part. - - -Use PyTorch iOS library with Metal ----------------------------------- -The PyTorch iOS library with Metal support ``LibTorch-Lite-Nightly`` is available in Cocoapods. You can read the `Using the Nightly PyTorch iOS Libraries in CocoaPods `_ section from the iOS tutorial for more detail about its usage. - -We also have the `HelloWorld-Metal example `_ that shows how to conect all pieces together. - -Note that if you run the HelloWorld-Metal example, you may notice that the results are slighly different from the `results `_ we got from the CPU model as shown in the iOS tutorial. - -.. code:: shell - - - timber wolf, grey wolf, gray wolf, Canis lupus - - malamute, malemute, Alaskan malamute - - Eskimo dog, husky - -This is because by default Metal uses fp16 rather than fp32 to compute. The precision loss is expected. - - -Use LibTorch-Lite Built from Source ------------------------------------ - -You can also build a custom LibTorch-Lite from Source and use it to run GPU models on iOS Metal. In this section, we'll be using the `HelloWorld example `_ to demonstrate this process. - -First, make sure you have deleted the **build** folder from the "Model Preparation" step in PyTorch root directory. Then run the command below - -.. code:: shell - - IOS_ARCH=arm64 USE_PYTORCH_METAL=1 ./scripts/build_ios.sh - -Note ``IOS_ARCH`` tells the script to build a arm64 version of Libtorch-Lite. This is because in PyTorch, Metal is only available for the iOS devices that support the Apple A9 chip or above. Once the build finished, follow the `Build PyTorch iOS libraries from source `_ section from the iOS tutorial to setup the XCode settings properly. Don't forget to copy the ``./mobilenetv2_metal.pt`` to your XCode project and modify the model file path accordingly. - -Next we need to make some changes in ``TorchModule.mm`` - -.. code:: objective-c - - ... - // #import - // If it's built from source with Xcode, comment out the line above - // and use following headers - #include - #include - #include - ... - - - (NSArray*)predictImage:(void*)imageBuffer { - c10::InferenceMode mode; - at::Tensor tensor = torch::from_blob(imageBuffer, {1, 3, 224, 224}, at::kFloat).metal(); - auto outputTensor = _impl.forward({tensor}).toTensor().cpu(); - ... - } - ... - -As you can see, we simply just call ``.metal()`` to move our input tensor from CPU to GPU, and then call ``.cpu()`` to move the result back. Internally, ``.metal()`` will copy the input data from the CPU buffer to a GPU buffer with a GPU compatible memory format. When ``.cpu()`` is invoked, the GPU command buffer will be flushed and synced. After `forward` finished, the final result will then be copied back from the GPU buffer back to a CPU buffer. - -The last step we have to do is to add the ``Accelerate.framework`` and the ``MetalPerformanceShaders.framework`` to your xcode project (Open your project via XCode, go to your project target’s "General" tab, locate the "Frameworks, Libraries and Embedded Content" section and click the "+" button). - -If everything works fine, you should be able to see the inference results on your phone. - - -Conclusion ----------- - -In this tutorial, we demonstrated how to convert a mobilenetv2 model to a GPU compatible model. We walked through a HelloWorld example to show how to use the C++ APIs to run models on iOS GPU. Please be aware of that GPU feature is still under development, new operators will continue to be added. APIs are subject to change in the future versions. - -Thanks for reading! As always, we welcome any feedback, so please create an issue `here `_ if you have any. - -Learn More ----------- - -- The `Mobilenetv2 `_ from Torchvision -- To learn more about how to use ``optimize_for_mobile``, please refer to the `Mobile Perf Recipe `_ + diff --git a/prototype_source/nnapi_mobilenetv2.rst b/prototype_source/nnapi_mobilenetv2.rst index ed9548a387d..ef7edc92d12 100644 --- a/prototype_source/nnapi_mobilenetv2.rst +++ b/prototype_source/nnapi_mobilenetv2.rst @@ -1,218 +1,10 @@ (Beta) Convert MobileNetV2 to NNAPI ======================================== -Introduction ------------- +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -This tutorial shows how to prepare a computer vision model to use -`Android's Neural Networks API (NNAPI) `_. -NNAPI provides access to powerful and efficient computational cores -on many modern Android devices. +Redirecting in 3 seconds... -PyTorch's NNAPI is currently in the "prototype" phase and only supports -a limited range of operators, but we expect to solidify the integration -and expand our operator support over time. +.. raw:: html - -Environment ------------ - -Install PyTorch and torchvision. - -``pip install torch==1.10.0 torchvision==0.11.1`` - - -Model Preparation ------------------ - -First, we must prepare our model to execute with NNAPI. -This step runs on your training server or laptop. -The key conversion function to call is -``torch.backends._nnapi.prepare.convert_model_to_nnapi``, -but some extra steps are required to ensure that -the model is properly structured. -Most notably, quantizing the model is required -in order to run the model on certain accelerators. - -You can copy/paste this entire Python script and run it, -or make your own modifications. -By default, it will save the models to ``~/mobilenetv2-nnapi/``. -Please create that directory first. - -.. code:: python - - #!/usr/bin/env python - import sys - import os - import torch - import torch.utils.bundled_inputs - import torch.utils.mobile_optimizer - import torch.backends._nnapi.prepare - import torchvision.models.quantization.mobilenet - from pathlib import Path - - - # This script supports 3 modes of quantization: - # - "none": Fully floating-point model. - # - "core": Quantize the core of the model, but wrap it a - # quantizer/dequantizer pair, so the interface uses floating point. - # - "full": Quantize the model, and use quantized tensors - # for input and output. - # - # "none" maintains maximum accuracy - # "core" sacrifices some accuracy for performance, - # but maintains the same interface. - # "full" maximized performance (with the same accuracy as "core"), - # but requires the application to use quantized tensors. - # - # There is a fourth option, not supported by this script, - # where we include the quant/dequant steps as NNAPI operators. - def make_mobilenetv2_nnapi(output_dir_path, quantize_mode): - quantize_core, quantize_iface = { - "none": (False, False), - "core": (True, False), - "full": (True, True), - }[quantize_mode] - - model = torchvision.models.quantization.mobilenet.mobilenet_v2(pretrained=True, quantize=quantize_core) - model.eval() - - # Fuse BatchNorm operators in the floating point model. - # (Quantized models already have this done.) - # Remove dropout for this inference-only use case. - if not quantize_core: - model.fuse_model() - assert type(model.classifier[0]) == torch.nn.Dropout - model.classifier[0] = torch.nn.Identity() - - input_float = torch.zeros(1, 3, 224, 224) - input_tensor = input_float - - # If we're doing a quantized model, we need to trace only the quantized core. - # So capture the quantizer and dequantizer, use them to prepare the input, - # and replace them with identity modules so we can trace without them. - if quantize_core: - quantizer = model.quant - dequantizer = model.dequant - model.quant = torch.nn.Identity() - model.dequant = torch.nn.Identity() - input_tensor = quantizer(input_float) - - # Many NNAPI backends prefer NHWC tensors, so convert our input to channels_last, - # and set the "nnapi_nhwc" attribute for the converter. - input_tensor = input_tensor.contiguous(memory_format=torch.channels_last) - input_tensor.nnapi_nhwc = True - - # Trace the model. NNAPI conversion only works with TorchScript models, - # and traced models are more likely to convert successfully than scripted. - with torch.no_grad(): - traced = torch.jit.trace(model, input_tensor) - nnapi_model = torch.backends._nnapi.prepare.convert_model_to_nnapi(traced, input_tensor) - - # If we're not using a quantized interface, wrap a quant/dequant around the core. - if quantize_core and not quantize_iface: - nnapi_model = torch.nn.Sequential(quantizer, nnapi_model, dequantizer) - model.quant = quantizer - model.dequant = dequantizer - # Switch back to float input for benchmarking. - input_tensor = input_float.contiguous(memory_format=torch.channels_last) - - # Optimize the CPU model to make CPU-vs-NNAPI benchmarks fair. - model = torch.utils.mobile_optimizer.optimize_for_mobile(torch.jit.script(model)) - - # Bundle sample inputs with the models for easier benchmarking. - # This step is optional. - class BundleWrapper(torch.nn.Module): - def __init__(self, mod): - super().__init__() - self.mod = mod - def forward(self, arg): - return self.mod(arg) - nnapi_model = torch.jit.script(BundleWrapper(nnapi_model)) - torch.utils.bundled_inputs.augment_model_with_bundled_inputs( - model, [(torch.utils.bundled_inputs.bundle_large_tensor(input_tensor),)]) - torch.utils.bundled_inputs.augment_model_with_bundled_inputs( - nnapi_model, [(torch.utils.bundled_inputs.bundle_large_tensor(input_tensor),)]) - - # Save both models. - model._save_for_lite_interpreter(str(output_dir_path / ("mobilenetv2-quant_{}-cpu.pt".format(quantize_mode)))) - nnapi_model._save_for_lite_interpreter(str(output_dir_path / ("mobilenetv2-quant_{}-nnapi.pt".format(quantize_mode)))) - - - if __name__ == "__main__": - for quantize_mode in ["none", "core", "full"]: - make_mobilenetv2_nnapi(Path(os.environ["HOME"]) / "mobilenetv2-nnapi", quantize_mode) - - -Running Benchmarks ------------------- - -Now that the models are ready, we can benchmark them on our Android devices. -See `our performance recipe `_ for details. -The best-performing models are likely to be the "fully-quantized" models: -``mobilenetv2-quant_full-cpu.pt`` and ``mobilenetv2-quant_full-nnapi.pt``. - -Because these models have bundled inputs, we can run the benchmark as follows: - -.. code:: shell - - ./speed_benchmark_torch --pthreadpool_size=1 --model=mobilenetv2-quant_full-nnapi.pt --use_bundled_input=0 --warmup=5 --iter=200 - -Adjusting increasing the thread pool size can can reduce latency, -at the cost of increased CPU usage. -Omitting that argument will use one thread per big core. -The CPU models can get improved performance (at the cost of memory usage) -by passing ``--use_caching_allocator=true``. - - -Running model on host ---------------------- - -We can now run models on your linux machine using the reference implementation -of NNAPI. You need to build the NNAPI library from Android source code: - -* Make sure you have at least 200GB of disk space -* Follow `these instructions `_ to install ``repo`` - -.. code:: shell - - mkdir ~/android-nnapi && cd ~/android-nnapi - repo init -u https://android.googlesource.com/platform/manifest -b master - repo sync --network-only -j 16 - repo sync -l - . build/envsetup.sh - lunch aosp_x86_64-eng - mm -j16 out/host/linux-x86/lib64/libneuralnetworks.so - - -With the host build of ``libneuralnetworks.so`` you can run Pytorch NNAPI models on -your linux machine: - -.. code:: python - - #!/usr/bin/env python - import ctypes - import torch - from pathlib import Path - - ctypes.cdll.LoadLibrary(Path.home() / "android-nnapi/out/host/linux-x86/lib64/libneuralnetworks.so") - model = torch.jit.load(Path.home() / "mobilenetv2-nnapi/mobilenetv2-quant_full-nnapi.pt") - print(model(*model.get_all_bundled_inputs()[0])) - - -Integration ------------ - -The converted models are ordinary TorchScript models. -You can use them in your app just like any other PyTorch model. -See `https://pytorch.org/mobile/android/ `_ -for an introduction to using PyTorch on Android. - - -Learn More ----------- - -- Learn more about optimization in our - `Mobile Performance Recipe `_ -- `MobileNetV2 `_ from torchvision -- Information about `NNAPI `_ + diff --git a/recipes_source/mobile_interpreter.rst b/recipes_source/mobile_interpreter.rst index 44036e74ffd..e6d2056e1a6 100644 --- a/recipes_source/mobile_interpreter.rst +++ b/recipes_source/mobile_interpreter.rst @@ -1,201 +1,10 @@ (beta) Efficient mobile interpreter in Android and iOS ================================================================== -**Author**: `Chen Lai `_, `Martin Yuan `_ +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -.. warning:: - PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `_, PyTorch’s all-new on-device inference library. You can also review our new documentation to learn more about how to build `iOS `_ and `Android `_ apps with ExecuTorch. +Redirecting in 3 seconds... -Introduction ------------- +.. raw:: html -This tutorial introduces the steps to use PyTorch's efficient interpreter on iOS and Android. We will be using an Image Segmentation demo application as an example. - -This application will take advantage of the pre-built interpreter libraries available for Android and iOS, which can be used directly with Maven (Android) and CocoaPods (iOS). It is important to note that the pre-built libraries are the available for simplicity, but further size optimization can be achieved with by utilizing PyTorch's custom build capabilities. - -.. note:: If you see the error message: `PytorchStreamReader failed locating file bytecode.pkl: file not found ()`, likely you are using a torch script model that requires the use of the PyTorch JIT interpreter (a version of our PyTorch interpreter that is not as size-efficient). In order to leverage our efficient interpreter, please regenerate the model by running: `module._save_for_lite_interpreter(${model_path})`. - - - If `bytecode.pkl` is missing, likely the model is generated with the api: `module.save(${model_psth})`. - - The api `_load_for_lite_interpreter(${model_psth})` can be helpful to validate model with the efficient mobile interpreter. - -Android -------------------- -Get the Image Segmentation demo app in Android: https://github.com/pytorch/android-demo-app/tree/master/ImageSegmentation - -1. **Prepare model**: Prepare the mobile interpreter version of model by run the script below to generate the scripted model `deeplabv3_scripted.pt` and `deeplabv3_scripted.ptl` - -.. code:: python - - import torch - from torch.utils.mobile_optimizer import optimize_for_mobile - model = torch.hub.load('pytorch/vision:v0.7.0', 'deeplabv3_resnet50', pretrained=True) - model.eval() - - scripted_module = torch.jit.script(model) - # Export full jit version model (not compatible mobile interpreter), leave it here for comparison - scripted_module.save("deeplabv3_scripted.pt") - # Export mobile interpreter version model (compatible with mobile interpreter) - optimized_scripted_module = optimize_for_mobile(scripted_module) - optimized_scripted_module._save_for_lite_interpreter("deeplabv3_scripted.ptl") - -2. **Use the PyTorch Android library in the ImageSegmentation app**: Update the `dependencies` part of ``ImageSegmentation/app/build.gradle`` to - -.. code:: gradle - - repositories { - maven { - url "https://oss.sonatype.org/content/repositories/snapshots" - } - } - - dependencies { - implementation 'androidx.appcompat:appcompat:1.2.0' - implementation 'androidx.constraintlayout:constraintlayout:2.0.2' - testImplementation 'junit:junit:4.12' - androidTestImplementation 'androidx.test.ext:junit:1.1.2' - androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0' - implementation 'org.pytorch:pytorch_android_lite:1.9.0' - implementation 'org.pytorch:pytorch_android_torchvision:1.9.0' - - implementation 'com.facebook.fbjni:fbjni-java-only:0.0.3' - } - - - -3. **Update model loader api**: Update ``ImageSegmentation/app/src/main/java/org/pytorch/imagesegmentation/MainActivity.java`` by - - 4.1 Add new import: `import org.pytorch.LiteModuleLoader` - - 4.2 Replace the way to load pytorch lite model - -.. code:: java - - // mModule = Module.load(MainActivity.assetFilePath(getApplicationContext(), "deeplabv3_scripted.pt")); - mModule = LiteModuleLoader.load(MainActivity.assetFilePath(getApplicationContext(), "deeplabv3_scripted.ptl")); - -4. **Test app**: Build and run the `ImageSegmentation` app in Android Studio - -iOS -------------------- -Get ImageSegmentation demo app in iOS: https://github.com/pytorch/ios-demo-app/tree/master/ImageSegmentation - -1. **Prepare model**: Same as Android. - -2. **Build the project with Cocoapods and prebuilt interpreter** Update the `PodFile` and run `pod install`: - -.. code-block:: podfile - - target 'ImageSegmentation' do - # Comment the next line if you don't want to use dynamic frameworks - use_frameworks! - - # Pods for ImageSegmentation - pod 'LibTorch_Lite', '~>1.9.0' - end - -3. **Update library and API** - - 3.1 Update ``TorchModule.mm``: To use the custom built libraries project, use `` (in ``TorchModule.mm``): - -.. code-block:: swift - - #import - // If it's built from source with xcode, comment out the line above - // and use following headers - // #include - // #include - // #include - -.. code-block:: swift - - @implementation TorchModule { - @protected - // torch::jit::script::Module _impl; - torch::jit::mobile::Module _impl; - } - - - (nullable instancetype)initWithFileAtPath:(NSString*)filePath { - self = [super init]; - if (self) { - try { - _impl = torch::jit::_load_for_mobile(filePath.UTF8String); - // _impl = torch::jit::load(filePath.UTF8String); - // _impl.eval(); - } catch (const std::exception& exception) { - NSLog(@"%s", exception.what()); - return nil; - } - } - return self; - } - -3.2 Update ``ViewController.swift`` - -.. code-block:: swift - - // if let filePath = Bundle.main.path(forResource: - // "deeplabv3_scripted", ofType: "pt"), - // let module = TorchModule(fileAtPath: filePath) { - // return module - // } else { - // fatalError("Can't find the model file!") - // } - if let filePath = Bundle.main.path(forResource: - "deeplabv3_scripted", ofType: "ptl"), - let module = TorchModule(fileAtPath: filePath) { - return module - } else { - fatalError("Can't find the model file!") - } - -4. Build and test the app in Xcode. - -How to use mobile interpreter + custom build ---------------------------------------------- -A custom PyTorch interpreter library can be created to reduce binary size, by only containing the operators needed by the model. In order to do that follow these steps: - -1. To dump the operators in your model, say `deeplabv3_scripted`, run the following lines of Python code: - -.. code-block:: python - - # Dump list of operators used by deeplabv3_scripted: - import torch, yaml - model = torch.jit.load('deeplabv3_scripted.ptl') - ops = torch.jit.export_opnames(model) - with open('deeplabv3_scripted.yaml', 'w') as output: - yaml.dump(ops, output) - -In the snippet above, you first need to load the ScriptModule. Then, use export_opnames to return a list of operator names of the ScriptModule and its submodules. Lastly, save the result in a yaml file. The yaml file can be generated for any PyTorch 1.4.0 or above version. You can do that by checking the value of `torch.__version__`. - -2. To run the build script locally with the prepared yaml list of operators, pass in the yaml file generate from the last step into the environment variable SELECTED_OP_LIST. Also in the arguments, specify BUILD_PYTORCH_MOBILE=1 as well as the platform/architechture type. - -**iOS**: Take the simulator build for example, the command should be: - -.. code-block:: bash - - SELECTED_OP_LIST=deeplabv3_scripted.yaml BUILD_PYTORCH_MOBILE=1 IOS_PLATFORM=SIMULATOR ./scripts/build_ios.sh - -**Android**: Take the x86 build for example, the command should be: - -.. code-block:: bash - - SELECTED_OP_LIST=deeplabv3_scripted.yaml ./scripts/build_pytorch_android.sh x86 - - - -Conclusion ----------- - -In this tutorial, we demonstrated how to use PyTorch's efficient mobile interpreter, in an Android and iOS app. - -We walked through an Image Segmentation example to show how to dump the model, build a custom torch library from source and use the new api to run model. - -Our efficient mobile interpreter is still under development, and we will continue improving its size in the future. Note, however, that the APIs are subject to change in future versions. - -Thanks for reading! As always, we welcome any feedback, so please create an issue `here ` - if you have any. - -Learn More ----------- - -- To learn more about PyTorch Mobile, please refer to `PyTorch Mobile Home Page `_ -- To learn more about Image Segmentation, please refer to the `Image Segmentation DeepLabV3 on Android Recipe `_ + diff --git a/recipes_source/mobile_perf.rst b/recipes_source/mobile_perf.rst index 14f183ab69e..8835ddecc6d 100644 --- a/recipes_source/mobile_perf.rst +++ b/recipes_source/mobile_perf.rst @@ -1,359 +1,10 @@ Pytorch Mobile Performance Recipes ================================== -.. warning:: - PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `_, PyTorch’s all-new on-device inference library. You can also learn more about `quantization `_, `Hardware acceleration (op fusion using hw) `_, and `benchmarking `_ on ExecuTorch’s documentation pages. +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -Introduction ----------------- -Performance (aka latency) is crucial to most, if not all, -applications and use-cases of ML model inference on mobile devices. +Redirecting in 3 seconds... -Today, PyTorch executes the models on the CPU backend pending availability -of other hardware backends such as GPU, DSP, and NPU. +.. raw:: html -In this recipe, you will learn: - -- How to optimize your model to help decrease execution time (higher performance, lower latency) on the mobile device. -- How to benchmark (to check if optimizations helped your use case). - - -Model preparation ------------------ - -We will start with preparing to optimize your model to help decrease execution time -(higher performance, lower latency) on the mobile device. - - -Setup -^^^^^^^ - -First we need to installed pytorch using conda or pip with version at least 1.5.0. - -:: - - conda install pytorch torchvision -c pytorch - -or - -:: - - pip install torch torchvision - -Code your model: - -:: - - import torch - from torch.utils.mobile_optimizer import optimize_for_mobile - - class AnnotatedConvBnReLUModel(torch.nn.Module): - def __init__(self): - super(AnnotatedConvBnReLUModel, self).__init__() - self.conv = torch.nn.Conv2d(3, 5, 3, bias=False).to(dtype=torch.float) - self.bn = torch.nn.BatchNorm2d(5).to(dtype=torch.float) - self.relu = torch.nn.ReLU(inplace=True) - self.quant = torch.quantization.QuantStub() - self.dequant = torch.quantization.DeQuantStub() - - def forward(self, x): - x = x.contiguous(memory_format=torch.channels_last) - x = self.quant(x) - x = self.conv(x) - x = self.bn(x) - x = self.relu(x) - x = self.dequant(x) - return x - - model = AnnotatedConvBnReLUModel() - - -``torch.quantization.QuantStub`` and ``torch.quantization.DeQuantStub()`` are no-op stubs, which will be used for quantization step. - - -1. Fuse operators using ``torch.quantization.fuse_modules`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Do not be confused that fuse_modules is in the quantization package. -It works for all ``torch.nn.Module``. - -``torch.quantization.fuse_modules`` fuses a list of modules into a single module. -It fuses only the following sequence of modules: - -- Convolution, Batch normalization -- Convolution, Batch normalization, Relu -- Convolution, Relu -- Linear, Relu - -This script will fuse Convolution, Batch Normalization and Relu in previously declared model. - -:: - - torch.quantization.fuse_modules(model, [['conv', 'bn', 'relu']], inplace=True) - - -2. Quantize your model -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -You can find more about PyTorch quantization in -`the dedicated tutorial `_. - -Quantization of the model not only moves computation to int8, -but also reduces the size of your model on a disk. -That size reduction helps to reduce disk read operations during the first load of the model and decreases the amount of RAM. -Both of those resources can be crucial for the performance of mobile applications. -This code does quantization, using stub for model calibration function, you can find more about it `here `__. - -:: - - model.qconfig = torch.quantization.get_default_qconfig('qnnpack') - torch.quantization.prepare(model, inplace=True) - # Calibrate your model - def calibrate(model, calibration_data): - # Your calibration code here - return - calibrate(model, []) - torch.quantization.convert(model, inplace=True) - - - -3. Use torch.utils.mobile_optimizer -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Torch mobile_optimizer package does several optimizations with the scripted model, -which will help to conv2d and linear operations. -It pre-packs model weights in an optimized format and fuses ops above with relu -if it is the next operation. - -First we script the result model from previous step: - -:: - - torchscript_model = torch.jit.script(model) - -Next we call ``optimize_for_mobile`` and save model on the disk. - -:: - - torchscript_model_optimized = optimize_for_mobile(torchscript_model) - torch.jit.save(torchscript_model_optimized, "model.pt") - -4. Prefer Using Channels Last Tensor memory format -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Channels Last(NHWC) memory format was introduced in PyTorch 1.4.0. It is supported only for four-dimensional tensors. This memory format gives a better memory locality for most operators, especially convolution. Our measurements showed a 3x speedup of MobileNetV2 model compared with the default Channels First(NCHW) format. - -At the moment of writing this recipe, PyTorch Android java API does not support using inputs in Channels Last memory format. But it can be used on the TorchScript model level, by adding the conversion to it for model inputs. - -.. code-block:: python - - def forward(self, x): - x = x.contiguous(memory_format=torch.channels_last) - ... - - -This conversion is zero cost if your input is already in Channels Last memory format. After it, all operators will work preserving ChannelsLast memory format. - -5. Android - Reusing tensors for forward -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This part of the recipe is Android only. - -Memory is a critical resource for android performance, especially on old devices. -Tensors can need a significant amount of memory. -For example, standard computer vision tensor contains 1*3*224*224 elements, -assuming that data type is float and will need 588Kb of memory. - -:: - - FloatBuffer buffer = Tensor.allocateFloatBuffer(1*3*224*224); - Tensor tensor = Tensor.fromBlob(buffer, new long[]{1, 3, 224, 224}); - - -Here we allocate native memory as ``java.nio.FloatBuffer`` and creating ``org.pytorch.Tensor`` which storage will be pointing to the memory of the allocated buffer. - -For most of the use cases, we do not do model forward only once, repeating it with some frequency or as fast as possible. - -If we are doing new memory allocation for every module forward - that will be suboptimal. -Instead of this, we can reuse the same memory that we allocated on the previous step, fill it with new data, and run module forward again on the same tensor object. - -You can check how it looks in code in `pytorch android application example `_. - -:: - - protected AnalysisResult analyzeImage(ImageProxy image, int rotationDegrees) { - if (mModule == null) { - mModule = Module.load(moduleFileAbsoluteFilePath); - mInputTensorBuffer = - Tensor.allocateFloatBuffer(3 * 224 * 224); - mInputTensor = Tensor.fromBlob(mInputTensorBuffer, new long[]{1, 3, 224, 224}); - } - - TensorImageUtils.imageYUV420CenterCropToFloatBuffer( - image.getImage(), rotationDegrees, - 224, 224, - TensorImageUtils.TORCHVISION_NORM_MEAN_RGB, - TensorImageUtils.TORCHVISION_NORM_STD_RGB, - mInputTensorBuffer, 0); - - Tensor outputTensor = mModule.forward(IValue.from(mInputTensor)).toTensor(); - } - -Member fields ``mModule``, ``mInputTensorBuffer`` and ``mInputTensor`` are initialized only once -and buffer is refilled using ``org.pytorch.torchvision.TensorImageUtils.imageYUV420CenterCropToFloatBuffer``. - -6. Load time optimization -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -**Available since Pytorch 1.13** - -PyTorch Mobile also supports a FlatBuffer-based file format that is faster -to load. Both flatbuffer and pickle-based model file can be load with the -same ``_load_for_lite_interpreter`` (Python) or ``_load_for_mobile``(C++) API. - -To use the FlatBuffer format, instead of creating the model file with -``model._save_for_lite_interpreter('path/to/file.ptl')``, you can run the following command: - - -One can save using - -:: - - model._save_for_lite_interpreter('path/to/file.ptl', _use_flatbuffer=True) - - -The extra argument ``_use_flatbuffer`` makes a FlatBuffer file instead of a -zip file. The created file will be faster to load. - -For example, using ResNet-50 and running the following script: - -:: - - import torch - from torch.jit import mobile - import time - model = torch.hub.load('pytorch/vision:v0.10.0', 'deeplabv3_resnet50', pretrained=True) - model.eval() - jit_model = torch.jit.script(model) - - jit_model._save_for_lite_interpreter('/tmp/jit_model.ptl') - jit_model._save_for_lite_interpreter('/tmp/jit_model.ff', _use_flatbuffer=True) - - import timeit - print('Load ptl file:') - print(timeit.timeit('from torch.jit import mobile; mobile._load_for_lite_interpreter("/tmp/jit_model.ptl")', - number=20)) - print('Load flatbuffer file:') - print(timeit.timeit('from torch.jit import mobile; mobile._load_for_lite_interpreter("/tmp/jit_model.ff")', - number=20)) - - - -you would get the following result: - -:: - - Load ptl file: - 0.5387594579999999 - Load flatbuffer file: - 0.038842832999999466 - -While speed ups on actual mobile devices will be smaller, you can still expect -3x - 6x load time reductions. - -### Reasons to avoid using a FlatBuffer-based mobile model - -However, FlatBuffer format also has some limitations that you might want to consider: - -* It is only available in PyTorch 1.13 or later. Therefore, client devices compiled - with earlier PyTorch versions might not be able to load it. -* The Flatbuffer library imposes a 4GB limit for file sizes. So it is not suitable - for large models. - -Benchmarking ------------- - -The best way to benchmark (to check if optimizations helped your use case) - is to measure your particular use case that you want to optimize, as performance behavior can vary in different environments. - -PyTorch distribution provides a way to benchmark naked binary that runs the model forward, -this approach can give more stable measurements rather than testing inside the application. - - -Android - Benchmarking Setup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This part of the recipe is Android only. - -For this you first need to build benchmark binary: - -:: - - - rm -rf build_android - BUILD_PYTORCH_MOBILE=1 ANDROID_ABI=arm64-v8a ./scripts/build_android.sh -DBUILD_BINARY=ON - -You should have arm64 binary at: ``build_android/bin/speed_benchmark_torch``. -This binary takes ``--model=``, ``--input_dim="1,3,224,224"`` as dimension information for the input and ``--input_type="float"`` as the type of the input as arguments. - -Once you have your android device connected, -push speedbenchark_torch binary and your model to the phone: - -:: - - adb push /data/local/tmp - adb push /data/local/tmp - - -Now we are ready to benchmark your model: - -:: - - adb shell "/data/local/tmp/speed_benchmark_torch --model=/data/local/tmp/model.pt" --input_dims="1,3,224,224" --input_type="float" - ----- output ----- - Starting benchmark. - Running warmup runs. - Main runs. - Main run finished. Microseconds per iter: 121318. Iters per second: 8.24281 - - -iOS - Benchmarking Setup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For iOS, we'll be using our `TestApp `_ as the benchmarking tool. - -To begin with, let's apply the ``optimize_for_mobile`` method to our python script located at `TestApp/benchmark/trace_model.py `_. Simply modify the code as below. - -:: - - import torch - import torchvision - from torch.utils.mobile_optimizer import optimize_for_mobile - - model = torchvision.models.mobilenet_v2(pretrained=True) - model.eval() - example = torch.rand(1, 3, 224, 224) - traced_script_module = torch.jit.trace(model, example) - torchscript_model_optimized = optimize_for_mobile(traced_script_module) - torch.jit.save(torchscript_model_optimized, "model.pt") - -Now let's run ``python trace_model.py``. If everything works well, we should be able to generate our optimized model in the benchmark directory. - -Next, we're going to build the PyTorch libraries from source. - -:: - - BUILD_PYTORCH_MOBILE=1 IOS_ARCH=arm64 ./scripts/build_ios.sh - -Now that we have the optimized model and PyTorch ready, it's time to generate our XCode project and do benchmarking. To do that, we'll be using a ruby script - `setup.rb` which does the heavy lifting jobs of setting up the XCode project. - -:: - - ruby setup.rb - -Now open the `TestApp.xcodeproj` and plug in your iPhone, you're ready to go. Below is an example result from iPhoneX - -:: - - TestApp[2121:722447] Main runs - TestApp[2121:722447] Main run finished. Milliseconds per iter: 28.767 - TestApp[2121:722447] Iters per second: : 34.762 - TestApp[2121:722447] Done. + diff --git a/recipes_source/model_preparation_android.rst b/recipes_source/model_preparation_android.rst index 55ef7d9735c..22c0e17df31 100644 --- a/recipes_source/model_preparation_android.rst +++ b/recipes_source/model_preparation_android.rst @@ -1,85 +1,10 @@ Model Preparation for Android Recipe ===================================== -This recipe demonstrates how to prepare a PyTorch MobileNet v2 image classification model for Android apps, and how to set up Android projects to use the mobile-ready model file. +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -Introduction ------------------ +Redirecting in 3 seconds... -After a PyTorch model is trained or a pre-trained model is made available, it is normally not ready to be used in mobile apps yet. It needs to be quantized (see the `Quantization Recipe `_), converted to TorchScript so Android apps can load it, and optimized for mobile apps. Furthermore, Android apps need to be set up correctly to enable the use of PyTorch Mobile libraries, before they can load and use the model for inference. +.. raw:: html -Pre-requisites ------------------ - -PyTorch 1.6.0 or 1.7.0 - -torchvision 0.6.0 or 0.7.0 - -Android Studio 3.5.1 or above with NDK installed - -Steps ------------------ - -1. Get Pretrained and Quantized MobileNet v2 Model -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To get the MobileNet v2 quantized model, simply do: - -:: - - import torchvision - - model_quantized = torchvision.models.quantization.mobilenet_v2(pretrained=True, quantize=True) - -2. Script and Optimize the Model for Mobile Apps -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Use either the `script` or `trace` method to convert the quantized model to the TorchScript format: - -:: - - import torch - - dummy_input = torch.rand(1, 3, 224, 224) - torchscript_model = torch.jit.trace(model_quantized, dummy_input) - -or - -:: - - torchscript_model = torch.jit.script(model_quantized) - - -.. warning:: - The `trace` method only scripts the code path executed during the trace, so it will not work properly for models that include decision branches. See the `Script and Optimize for Mobile Recipe `_ for more details. - -Then optimize the TorchScript formatted model for mobile and save it: - -:: - - from torch.utils.mobile_optimizer import optimize_for_mobile - torchscript_model_optimized = optimize_for_mobile(torchscript_model) - torch.jit.save(torchscript_model_optimized, "mobilenetv2_quantized.pt") - -With the total 7 or 8 (depending on if the `script` or `trace` method is called to get the TorchScript format of the model) lines of code in the two steps above, we have a model ready to be added to mobile apps. - -3. Add the Model and PyTorch Library on Android -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -* In your current or a new Android Studio project, open the build.gradle file, and add the following two lines (the second one is required only if you plan to use a TorchVision model): - -:: - - implementation 'org.pytorch:pytorch_android:1.6.0' - implementation 'org.pytorch:pytorch_android_torchvision:1.6.0' - -* Drag and drop the model file `mobilenetv2_quantized.pt` to your project's assets folder. - -That's it! Now you can build your Android app with the PyTorch library and the model ready to use. To actually write code to use the model, refer to the PyTorch Mobile `Android Quickstart with a HelloWorld Example `_ and `Android Hackathon Example `_. - -Learn More ------------------ - -1. `PyTorch Mobile site `_ - -2. `Introduction to TorchScript `_ + diff --git a/recipes_source/model_preparation_ios.rst b/recipes_source/model_preparation_ios.rst index 2fbacd7fa68..cbb4927eaeb 100644 --- a/recipes_source/model_preparation_ios.rst +++ b/recipes_source/model_preparation_ios.rst @@ -1,95 +1,10 @@ Model Preparation for iOS Recipe ===================================== -This recipe demonstrates how to prepare a PyTorch MobileNet v2 image classification model for iOS apps, and how to set up an iOS project to use the mobile-ready model file. +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -Introduction ------------------ +Redirecting in 3 seconds... -After a PyTorch model is trained or a pre-trained model is made available, it is normally not ready to be used in mobile apps yet. It needs to be quantized (see `Quantization Recipe `_ for more details), converted to TorchScript so iOS apps can load it and optimized for mobile apps (see `Script and Optimize for Mobile Recipe `_). Furthermore, iOS apps need to be set up correctly to enable the use of PyTorch Mobile libraries, before they can load and use the model for inference. +.. raw:: html -Pre-requisites ------------------ - -PyTorch 1.6.0 or 1.7.0 - -torchvision 0.6.0 or 0.7.0 - -Xcode 11 or 12 - -Steps ------------------ - -1. Get Pretrained and Quantized MobileNet v2 Model -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To get the MobileNet v2 quantized model, simply do: - -:: - - import torchvision - - model_quantized = torchvision.models.quantization.mobilenet_v2(pretrained=True, quantize=True) - -2. Script and Optimize the Model for Mobile Apps -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Use either the script or trace method to convert the quantized model to the TorchScript format: - -:: - - import torch - - dummy_input = torch.rand(1, 3, 224, 224) - torchscript_model = torch.jit.trace(model_quantized, dummy_input) - -or - -:: - - torchscript_model = torch.jit.script(model_quantized) - -.. warning:: - The `trace` method only scripts the code path executed during the trace, so it will not work properly for models that include decision branches. See the `Script and Optimize for Mobile Recipe `_ for more details. - - -Then optimize the TorchScript formatted model for mobile and save it: - -:: - - from torch.utils.mobile_optimizer import optimize_for_mobile - torchscript_model_optimized = optimize_for_mobile(torchscript_model) - torch.jit.save(torchscript_model_optimized, "mobilenetv2_quantized.pt") - -With the total 7 or 8 (depending on if the script or trace method is called to get the TorchScript format of the model) lines of code in the two steps above, we have a model ready to be added to mobile apps. - -3. Add the Model and PyTorch Library on iOS -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -To use the mobile-ready model `mobilenetv2_quantized.pt` in an iOS app, either create a new Xcode project or in your existing Xcode project, then follow the steps below: - -* Open a Mac Terminal, cd to your iOS app's project folder; - -* If your iOS app does not use Cocoapods yet, run `pod init` first to generate the `Podfile` file. - -* Edit `Podfile` either from Xcode or any editor, and add the following line under the target: - -:: - - pod 'LibTorch', '~>1.6.1' - -* Run `pod install` from the Terminal and then open your project's xcworkspace file; - -* Save the two files `TorchModule.h` and `TorchModule.mm` from `here `_ and drag and drop them to your project. If your project is Swift based, a message box with the title "Would you like to configure an Objective-C bridging header?" will show up; click the "Create Bridging Header" button to create a Swift to Objective-c bridging header file, and add `#import "TorchModule.h"` to the header file `-Bridging-Header.h`; - -* Drag and drop the model file `mobilenetv2_quantized.pt` to the project. - -After these steps, you can successfully build and run your Xcode project. To actually write code to use the model, refer to the PyTorch Mobile `iOS Code Walkthrough `_ and two complete ready-to-run sample iOS apps `HelloWorld `_ and `iOS Hackathon Example `_. - - -Learn More ------------------ - -1. `PyTorch Mobile site `_ - -2. `Introduction to TorchScript `_ + diff --git a/recipes_source/ptmobile_recipes_summary.rst b/recipes_source/ptmobile_recipes_summary.rst index 6cc8f6f7514..fdf9f58e43d 100644 --- a/recipes_source/ptmobile_recipes_summary.rst +++ b/recipes_source/ptmobile_recipes_summary.rst @@ -1,43 +1,10 @@ Summary of PyTorch Mobile Recipes ===================================== -.. warning:: - Note: PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `_, PyTorch’s all-new on-device inference library. You can also review these `ExecuTorch examples `_. +PyTorch Mobile is no longer actively supported. Please check out `ExecuTorch `__. -This summary provides a top level overview of recipes for PyTorch Mobile to help developers choose which recipes to follow for their PyTorch-powered mobile app development. +Redirecting in 3 seconds... -Introduction ----------------- +.. raw:: html -When a PyTorch model is trained or retrained, or when a pre-trained model is available, for mobile deployment, follow the the recipes outlined in this summary so mobile apps can successfully use the model. - -Pre-requisites ----------------- - -PyTorch 1.6.0 or 1.7.0 - -(Optional) torchvision 0.6.0 or 0.7.0 - -For iOS development: Xcode 11 or 12 - -For Android development: Android Studio 3.5.1 or above (with NDK installed); or Android SDK, NDK, Gradle, JDK. - -New Recipes for PyTorch Mobile --------------------------------- - -* (Recommended) To fuse a list of PyTorch modules into a single module to reduce the model size before quantization, read the `Fuse Modules recipe `_. - -* (Recommended) To reduce the model size and make it run faster without losing much on accuracy, read the `Quantization Recipe `_. - -* (Must) To convert the model to TorchScipt and (optional) optimize it for mobile apps, read the `Script and Optimize for Mobile Recipe `_. - -* (Must for iOS development) To add the model in an iOS project and use PyTorch pod for iOS, read the `Model preparation for iOS Recipe `_. - -* (Must for Android development) To add the model in an Android project and use the PyTorch library for Android, read the `Model preparation for Android Recipe `_. - - -Learn More ------------------ - -1. `PyTorch Mobile site `_ -2. `PyTorch Mobile Performance Recipes `_ +