Add NNAPI tutorial #1229
Add NNAPI tutorial #1229
Conversation
|
Deploy preview for pytorch-tutorials-preview ready! Built with commit f8791e5 https://deploy-preview-1229--pytorch-tutorials-preview.netlify.app |
| NNAPI provides access to powerful and efficient computational cores | ||
| on many modern Android devices. | ||
|
|
||
| PyTorch's NNAPI is currently in the "prototype" phase and only supports |
There was a problem hiding this comment.
Should a developer who wants to try this prototype feature go to the PyTorch master branch?
There was a problem hiding this comment.
Updating to reference the specific version that is known to work.
| First, we must prepare our model to execute with NNAPI. | ||
| This step runs on your training server or laptop. | ||
| The key conversion function to call is | ||
| ``torch.backends._nnapi.prepare.convert_model_to_nnapi``, |
There was a problem hiding this comment.
Is there a link to the convert_model_to_nnapi documentation?
There was a problem hiding this comment.
No. This still needs to be written.
| but some extra steps are required to ensure that | ||
| the model is properly structured. | ||
| Most notably, quantizing the model is required | ||
| in order to run the model on certain hardware backends. |
There was a problem hiding this comment.
Can we be more specific on the hardware backends that can run a quantized model? Is quantization required to run on certain hardware backends but:
a. not required on others, or
b. on some other hardware backends, even quantizing a model won't make the model run?
There was a problem hiding this comment.
Clarified that this refers to accelerators.
| # - "none": Fully floating-point model. | ||
| # - "core": Quantize the core of the model, but wrap it a | ||
| # quantizer/dequantizer pair, so the interface uses floating point. | ||
| # - "full": Quantize the model, and use quantize tensors |
There was a problem hiding this comment.
quantize tensors => quantized tensors
| import torch | ||
| import torch.utils.bundled_inputs | ||
| import torch.utils.mobile_optimizer | ||
| import torch.backends._nnapi.prepare |
There was a problem hiding this comment.
Which PyTorch and torchvision versions are required to run this? I built and installed PyTorch from the master branch on Oct 22, got an error "ModuleNotFoundError: No module named 'torch.backends._nnapi'". Then I built from the latest pytorch master branch, and got a new error when running the script:
Exception: Unsupported node kind ('aten::size') in node %13 : int = aten::size(%input.88, %32) # /Users/jeffxtang/opt/anaconda3/lib/python3.8/site-packages/torchvision/models/mobilenet.py:166:0
I then installed torchvision from the latest source but the script still picks the older 0.7...
There was a problem hiding this comment.
Updated instructions to reference the specific known-working version.
|
|
||
| Because these models have bundled inputs, we can run the benchmark as follows: | ||
|
|
||
| .. code:: shell |
There was a problem hiding this comment.
The code below doesn't show in the preview, maybe due to some format issue. code => code-block?
|
|
||
| Now that the models are ready, we can benchmark them on our Android devices. | ||
| See `our performance recipe <https://pytorch.org/tutorials/recipes/mobile_perf.html#android-benchmarking-setup>`_ for details. | ||
| The best-performing models are likely to be the "fully-quantized" models: |
There was a problem hiding this comment.
it'd be great to show benchmark data here.. maybe in a future update.
There was a problem hiding this comment.
Yeah. I'll add this after the blog post goes live.
dreiss
force-pushed
the
dreiss-nnapi-tutorial
branch
from
4ea1ee0 to
f8791e5
Compare
No description provided.