Skip to content

Trying to integrate blog filters #1041

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: site
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions _includes/blog_post_nav.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<div class="vertical-blog-container">
<div class="col-md-4">
<p class="blog-date">{{ post.date | date: '%B %d, %Y' }}</p>
<h4>
<a href="{{ site.baseurl }}{{ post.url }}">{{ post.title }}</a>
</h4>
<p>{{ post.excerpt | remove: '<p>' | remove: '</p>' | truncate: 500}}</p>
<img src="{{ post.image }}" class="img-fluid">
</div>
<a href="{{ site.baseurl }}{{ post.url }}" class="btn btn-lg with-right-arrow">
Read More
</a>
</div>
85 changes: 85 additions & 0 deletions _includes/blog_tags_filter.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
{% if site.tags | size > 0 %}
{% capture _site_tags %}{% for tag in site.tags %}{{ tag | first }}{% unless forloop.last %}###{% endunless %}{% endfor %}{% endcapture %}
{% assign tags_alphabetically_sorted = _site_tags | split:'###' | sort %}
<script>
jQuery(document).ready(function () {
jQuery('.blog-filter-menu .blog-filter').click(function (e) {
jQuery(this).siblings().removeClass(['selected', 'all-tag-selected']);

if(jQuery(this).data("tag") == "all") {
jQuery(this).addClass("all-tag-selected");

jQuery('.blog-tag-container').fadeOut('fast', function () {
jQuery('.blog-posts-nav').fadeIn('fast');
jQuery('ul.pagination').fadeIn('fast');
});
} else {
jQuery(this).addClass("selected");

let this_obj = this;
jQuery('.blog-tag-container div[data-tag="'+jQuery(this).data("tag")+'"]').siblings().fadeOut('fast', function () {
jQuery('.blog-tag-container div[data-tag="'+jQuery(this_obj).data("tag")+'"]').fadeIn('fast');
});

jQuery('.blog-posts-nav, ul.pagination').fadeOut('fast', function () {
jQuery('.blog-tag-container').fadeIn('fast');
});
}
});
});
</script>
<style>
.blog-container, .blog-tag-container {
width: 100%;
}

.blog-container .blog-vertical .blog-filter-menu .filter-btn {
color: #979797;
border: 1px solid #979797;
display: inline-block;
text-align: center;
white-space: nowrap;
vertical-align: middle;
padding: 0.375rem 0.75rem;
font-size: 1rem;
line-height: 1.5;
margin-bottom: 5px;
}

.blog-container .blog-vertical .blog-filter-menu .all-tag-selected {
background-color: #979797;
color: #ffffff;
}

.blog-container .blog-vertical .blog-filter-menu .filter-btn.selected {
background-color: #e44c2c;
border: 1px solid #e44c2c;
color: #ffffff;
}

.blog-container .blog-vertical .blog-filter-menu .blog-filter {
cursor: pointer;
}

.blog-container .blog-vertical .blog-tag-container {
display: none;
}
</style>
<div class="blog-filter-menu">
<div class="blog-filter filter-btn all-tag-selected" data-tag="all">All</div>
{% for tag in tags_alphabetically_sorted %}
<div class="blog-filter filter-btn filter" data-tag="{{ tag }}">{{ tag }}</div>
{% endfor %}
</div>
<div class="blog-tag-container">
{% for tag in site.tags %}
<div data-tag="{{ tag[0] }}">
{% for post in tag[1] %}

{% include blog_post_nav.html %}

{% endfor %}
</div>
{% endfor %}
</div>
{% endif %}
32 changes: 12 additions & 20 deletions _layouts/blog.html
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@
<div class="main-background blog-background"></div>

{% assign posts = paginator.posts %}
{% assign display_post_categories = site.posts | map: 'categories' | join: ',' | replace: '-', ' ' | split: ',' | uniq | sort %}
{% assign current_page = page.url | downcase | remove: ".html" | split: '/' %}
{% assign post_categories = site.posts | map: 'categories' | join: ',' | split: ',' | uniq | sort %}
<!-- {% assign filtered_posts = site.posts | where: "category", "tag5" %} -->
<!-- {% assign display_post_categories = site.posts | map: 'categories' | join: ',' | replace: '-', ' ' | split: ',' | uniq | sort %} -->
<!-- {% assign current_page = page.url | downcase | remove: ".html" | split: '/' %} -->
<!-- {% assign post_categories = site.posts | map: 'categories' | join: ',' | split: ',' | uniq | sort %} -->

<div class="jumbotron jumbotron-fluid">
<div class="container">
Expand All @@ -32,24 +33,15 @@ <h1 class="blog-index-title">

<div class="main-content-wrapper">
<div class="main-content">
<div class="container">
<div class="container blog-container">
<div class="row blog-vertical">

{% for post in posts %}
<div class="vertical-blog-container">
<div class="col-md-4">
<p class="blog-date">{{ post.date | date: '%B %d, %Y' }}</p>
<h4>
<a href="{{ site.baseurl }}{{ post.url }}">{{ post.title }}</a>
</h4>
<p>{{ post.excerpt | remove: '<p>' | remove: '</p>' | truncate: 500}}</p>
<img src="{{ post.image }}" class="img-fluid">
</div>
<a href="{{ site.baseurl }}{{ post.url }}" class="btn btn-lg with-right-arrow">
Read More
</a>
</div>
{% endfor %}
{% include blog_tags_filter.html %}

<div class="blog-posts-nav">
{% for post in posts %}
{% include blog_post_nav.html %}
{% endfor %}
</div>
</div>

{% include pagination_buttons.html %}
Expand Down
59 changes: 34 additions & 25 deletions _posts/2022-3-16-running-pytorch-models-on-jetson-nano.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,26 @@
---
layout: blog_detail
title: 'Running PyTorch Models on Jetson Nano'
title: "Running PyTorch Models on Jetson Nano"
author: Jeff Tang, Hamid Shojanazeri, Geeta Chauhan
featured-img: 'assets/images/pytorch-logo.jpg'
featured-img: "assets/images/pytorch-logo.jpg"
tags:
- tag5
- tag6
- tag7
---

### Overview

NVIDIA [Jetson Nano](https://developer.nvidia.com/embedded/jetson-nano-developer-kit), part of the [Jetson family of products](https://developer.nvidia.com/embedded/jetson-modules) or Jetson modules, is a small yet powerful Linux (Ubuntu) based embedded computer with 2/4GB GPU. With it, you can run many PyTorch models efficiently. This document summarizes our experience of running different deep learning models using 3 different mechanisms on Jetson Nano:

1. Jetson Inference the higher-level NVIDIA API that has built-in support for running most common computer vision models which can be transfer-learned with PyTorch on the Jetson platform.
1. Jetson Inference the higher-level NVIDIA API that has built-in support for running most common computer vision models which can be transfer-learned with PyTorch on the Jetson platform.

2. TensorRT, an SDK for high-performance inference from NVIDIA that requires the conversion of a PyTorch model to ONNX, and then to the TensorRT engine file that the TensorRT runtime can run.
2. TensorRT, an SDK for high-performance inference from NVIDIA that requires the conversion of a PyTorch model to ONNX, and then to the TensorRT engine file that the TensorRT runtime can run.

3. PyTorch with the direct PyTorch API `torch.nn` for inference.
3. PyTorch with the direct PyTorch API `torch.nn` for inference.

### Setting up Jetson Nano

After purchasing a Jetson Nano [here](https://developer.nvidia.com/buy-jetson?product=jetson_nano&location=US), simply follow the clear step-by-step [instructions](https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit) to download and write the Jetson Nano Developer Kit SD Card Image to a microSD card, and complete the setup. After the setup is done and the Nano is booted, you’ll see the standard Linux prompt along with the username and the Nano name used in the setup.

To check the GPU status on Nano, run the following commands:
Expand Down Expand Up @@ -69,7 +75,8 @@ nvgstcapture-1.0 --orientation=2
```

### Using Jetson Inference
NVIDIA [Jetson Inference](https://github.com/dusty-nv/jetson-inference) API offers the easiest way to run image recognition, object detection, semantic segmentation, and pose estimation models on Jetson Nano. Jetson Inference has TensorRT built-in, so it’s very fast.

NVIDIA [Jetson Inference](https://github.com/dusty-nv/jetson-inference) API offers the easiest way to run image recognition, object detection, semantic segmentation, and pose estimation models on Jetson Nano. Jetson Inference has TensorRT built-in, so it’s very fast.

To test run Jetson Inference, first clone the repo and download the models:

Expand Down Expand Up @@ -104,13 +111,11 @@ $ ls -lt ~/jetson_inference/
-rw-r--r-- 1 root root 179760 Oct 15 21:29 jellyfish.jpg
```


<div style="display: flex; justify-content: space-between;">
<img src="/assets/images/blog-2022-3-10-using-jetson-interface-1.jpeg" alt="Using jest interface example 1" width="40%">
<img src="/assets/images/blog-2022-3-10-using-jetson-interface-2.jpeg" alt="Using jest interface example 2" width="60%">
</div>


<div style="display: flex; justify-content: space-between;">
<img src="/assets/images/blog-2022-3-10-using-jetson-interface-3.jpeg" alt="Using jest interface example 3" width="60%">
<img src="/assets/images/blog-2022-3-10-using-jetson-interface-4.jpeg" alt="Using jest interface example 4" width="40%">
Expand All @@ -128,8 +133,8 @@ torchvision (0.10.0a0+300a8a4)
Although Jetson Inference includes models already converted to the TensorRT engine file format, you can fine-tune the models by following the steps in Transfer Learning with PyTorch (for Jetson Inference) [here](https://github.com/dusty-nv/jetson-inference/blob/master/docs/pytorch-transfer-learning.md).

### Using TensorRT
[TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/) is an SDK for high-performance inference from NVIDIA. Jetson Nano supports TensorRT via the Jetpack SDK, included in the SD Card image used to set up Jetson Nano. To confirm that TensorRT is already installed in Nano, `run dpkg -l|grep -i tensorrt`:

[TensorRT](https://docs.nvidia.com/deeplearning/tensorrt/) is an SDK for high-performance inference from NVIDIA. Jetson Nano supports TensorRT via the Jetpack SDK, included in the SD Card image used to set up Jetson Nano. To confirm that TensorRT is already installed in Nano, `run dpkg -l|grep -i tensorrt`:

<div class="text-center">
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-tensorrt.png" width="80%">
Expand All @@ -139,28 +144,29 @@ Theoretically, TensorRT can be used to “take a trained PyTorch model and optim

1. How to convert the model from PyTorch to ONNX;

2. How to convert the ONNX model to a TensorRT engine file;
2. How to convert the ONNX model to a TensorRT engine file;

3. How to run the engine file with the TensorRT runtime for performance improvement: inference time improved from the original 31.5ms/19.4ms (FP32/FP16 precision) to 6.28ms (TensorRT).

You can replace the Resnet50 model in the notebook code with another PyTorch model, go through the conversion process above, and run the finally converted model TensorRT engine file with the TensorRT runtime to see the optimized performance. But be aware that due to the Nano GPU memory size, models larger than 100MB are likely to fail to run, with the following error information:

`Error Code 1: Cuda Runtime (all CUDA-capable devices are busy or unavailable)`

You may also see an error when converting a PyTorch model to ONNX model, which may be fixed by replacing:
You may also see an error when converting a PyTorch model to ONNX model, which may be fixed by replacing:

`torch.onnx.export(resnet50, dummy_input, "resnet50_pytorch.onnx", verbose=False)`

with:

`torch.onnx.export(model, dummy_input, "deeplabv3_pytorch.onnx", opset_version=11, verbose=False)`

### Using PyTorch
### Using PyTorch

First, to download and install PyTorch 1.9 on Nano, run the following commands (see [here](https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-10-now-available/72048) for more information):

```
wget https://nvidia.box.com/shared/static/p57jwntv436lfrd78inwl7iml6p13fzh.whl -O torch-1.8.0-cp36-cp36m-linux_aarch64.whl -O torch-1.9.0-cp36-cp36m-linux_aarch64.whl
sudo apt-get install python3-pip libopenblas-base libopenmpi-dev
sudo apt-get install python3-pip libopenblas-base libopenmpi-dev
pip3 install Cython
pip3 install numpy torch-1.9.0-cp36-cp36m-linux_aarch64.whl
```
Expand All @@ -173,31 +179,32 @@ pip3 install torchvision-0.10.0a0+300a8a4-cp36-cp36m-linux_aarch64.whl
```

After the steps above, run this to confirm:

```
$ pip3 list|grep torch
torch (1.9.0)
torchvision (0.10.0)
```

You can also use the docker image described in the section *Using Jetson Inference* (which also has PyTorch and torchvision installed), to skip the manual steps above.
You can also use the docker image described in the section _Using Jetson Inference_ (which also has PyTorch and torchvision installed), to skip the manual steps above.

The official [YOLOv5](https://github.com/ultralytics/yolov5) repo is used to run the PyTorch YOLOv5 model on Jetson Nano. After logging in to Jetson Nano, follow the steps below:

* Get the repo and install what’s required:
- Get the repo and install what’s required:

```
git clone https://github.com/ultralytics/yolov5
cd yolov5
pip install -r requirements.txt
```

* Run `python3 detect.py`, which by default uses the PyTorch yolov5s.pt model. You should see something like:
- Run `python3 detect.py`, which by default uses the PyTorch yolov5s.pt model. You should see something like:

```
detect: weights=yolov5s.pt, source=data/images, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False
YOLOv5 🚀 v5.0-499-g48b00db torch 1.9.0 CUDA:0 (NVIDIA Tegra X1, 3956.1015625MB)

Fusing layers...
Fusing layers...
Model Summary: 224 layers, 7266973 parameters, 0 gradients
image 1/5 /home/jeff/repos/yolov5-new/yolov5/data/images/bus.jpg: 640x480 4 persons, 1 bus, 1 fire hydrant, Done. (0.142s)
...
Expand All @@ -206,6 +213,7 @@ image 1/5 /home/jeff/repos/yolov5-new/yolov5/data/images/bus.jpg: 640x480 4 pers
**The inference time on Jetson Nano GPU is about 140ms, more than twice as fast as the inference time on iOS or Android (about 330ms).**

If you get an error `“ImportError: The _imagingft C module is not installed.”` then you need to reinstall pillow:

```
sudo apt-get install libpng-dev
sudo apt-get install libfreetype6-dev
Expand All @@ -231,41 +239,42 @@ Using the same test files used in the PyTorch iOS YOLOv5 demo app or Android YOL
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-pytorch-1.png" alt="PyTorch YOLOv5 on Jetson Nano, example with a dog" width="35%">
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-pytorch-2.jpeg" alt="PyTorch YOLOv5 on Jetson Nano, example with a horse and a rider" width="50%">
</div>
Figure 1. PyTorch YOLOv5 on Jetson Nano.
Figure 1. PyTorch YOLOv5 on Jetson Nano.

<div style="display: flex">
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-pytorch-3.png" alt="PyTorch YOLOv5 on iOS, example with a dog" width="35%">
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-pytorch-4.png" alt="PyTorch YOLOv5 on iOS, example with a horse and a rider" width="50%">
</div>
Figure 2. PyTorch YOLOv5 on iOS.
Figure 2. PyTorch YOLOv5 on iOS.

<div style="display: flex">
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-pytorch-5.png" alt="PyTorch YOLOv5 on Android, example with a dog" width="35%">
<img src="{{ site.baseurl }}/assets/images/blog-2022-3-10-using-pytorch-6.png" alt="PyTorch YOLOv5 on Android, example with a horse and a rider" width="50%">
</div>
Figure 3. PyTorch YOLOv5 on Android.
Figure 3. PyTorch YOLOv5 on Android.

### Summary

Based on our experience of running different PyTorch models for potential demo apps on Jetson Nano, we see that even Jetson Nano, a lower-end of the Jetson family of products, provides a powerful GPU and embedded system that can directly run some of the latest PyTorch models, pre-trained or transfer learned, efficiently.

Building PyTorch demo apps on Jetson Nano can be similar to building PyTorch apps on Linux, but you can also choose to use TensorRT after converting the PyTorch models to the TensorRT engine file format.

But if you just need to run some common computer vision models on Jetson Nano using NVIDIA’s Jetson Inference which supports image recognition, object detection, semantic segmentation, and pose estimation models, then this is the easiest way.


### References

Torch-TensorRT, a compiler for PyTorch via TensorRT:
[https://github.com/NVIDIA/Torch-TensorRT/](https://github.com/NVIDIA/Torch-TensorRT/)

Jetson Inference docker image details:
[https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-docker.md](https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-docker.md)

A guide to using TensorRT on the NVIDIA Jetson Nano:
[https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/](https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/)
[https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/](https://docs.donkeycar.com/guide/robot_sbc/tensorrt_jetson_nano/)
including:

1. Use Jetson as a portable GPU device to run an NN chess engine model:
[https://medium.com/@ezchess/jetson-lc0-running-leela-chess-zero-on-nvidia-jetson-a-portable-gpu-device-a213afc9c018](https://medium.com/@ezchess/jetson-lc0-running-leela-chess-zero-on-nvidia-jetson-a-portable-gpu-device-a213afc9c018)
1. Use Jetson as a portable GPU device to run an NN chess engine model:
[https://medium.com/@ezchess/jetson-lc0-running-leela-chess-zero-on-nvidia-jetson-a-portable-gpu-device-a213afc9c018](https://medium.com/@ezchess/jetson-lc0-running-leela-chess-zero-on-nvidia-jetson-a-portable-gpu-device-a213afc9c018)

2. A MaskEraser app using PyTorch and torchvision, installed directly with pip:
[https://github.com/INTEC-ATI/MaskEraser#install-pytorch](https://github.com/INTEC-ATI/MaskEraser#install-pytorch)
[https://github.com/INTEC-ATI/MaskEraser#install-pytorch](https://github.com/INTEC-ATI/MaskEraser#install-pytorch)
Loading