Skip to content

Commit baea51e

Browse files
authored
Merge pull request #304 from arduino/nicla-vision-image-classification-tutorial-update
Nicla vision image classification tutorial update
2 parents d6ad2da + 9a03078 commit baea51e

File tree

2 files changed

+10
-4
lines changed
  • content/hardware/05.nicla/boards/nicla-vision/tutorials

2 files changed

+10
-4
lines changed

content/hardware/05.nicla/boards/nicla-vision/tutorials/getting-started/content.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ The OpenMV IDE is meant to provide an Arduino like experience for simple machine
3535

3636
Using the OpenMV IDE you can run [MicroPython](http://docs.MicroPython.org/en/latest/) scripts on the Nicla Vision board. MicroPython provides a lot of classes and modules that make it easy to quickly explore the features of the Nicla Vision. In this tutorial you will first download the OpenMV IDE and set up the development environment. [Here](https://openmv.io/) you can read more about the OpenMV IDE. OpenMV comes with its own firmware that is built on MicroPython. You will then learn to write a simple script that will blink the on-board RGB LED using some basic MicroPython commands.
3737

38-
***Before proceeding with the tutorial please update the board's bootloader. You can do this by first downloading the latest version of the "Mbed OS Nicla core" in the Arduino IDE. Then go to "File > Examples > STM32H747_System > STM32H747_updateBootloader" and upload this sketch to your board. After the sketch is uploaded follow the instructions in the serial monitor.***
38+
***Before proceeding with the tutorial please update the board's bootloader. You can do this by first downloading the latest version of the "Mbed OS Nicla core" in the Arduino IDE. Then go to "File > Examples > STM32H747_System > STM32H747_manageBootloader" and upload this sketch to your board. After the sketch is uploaded follow the instructions in the serial monitor.***
3939

4040
### 1. Downloading the OpenMV IDE
4141

content/hardware/05.nicla/boards/nicla-vision/tutorials/image-classification/content.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ To find the right configuration for your application often requires trial and er
5050

5151
The first step is to create a representative dataset of the objects that the ML model is supposed to identify. The key is to have as much diversity in the models as possible. If we show it for example only one specific apple that has a certain size, shape and peel, then it won't be very good at recognizing other apples that look different. This is referred to as a bias and should be avoided as much as possible. In addition you need to teach the model what an apple is not. For that purpose you feed it random image data of things that are not an apple. You could name that class of image data "unknown". If you don't have such a class and the model has only ever seen an apple, it won't know what to do if there is no apple in the image.
5252

53-
Creating data sets in OpenMV is simple as there is a built-in function to create them. Before you proceed, connect your Nicla Vision board. Click on the connect button in the OpenMV IDE. If you haven't set up your board for OpenMV please consult the [getting started tutorial](.\tutorials\getting-started).
53+
Creating data sets in OpenMV is simple as there is a built-in function to create them. Before you proceed, connect your Nicla Vision board. Click on the connect button in the OpenMV IDE. If you haven't set up your board for OpenMV please consult the [getting started tutorial](.\tutorials\nicla-vision\getting-started).
5454

5555
Create a new dataset by using the menu command **Tools > Dataset Editor > New Dataset** and name it `Dataset-Fruits`.
5656

@@ -147,7 +147,7 @@ Deploying the ML model to your board requires a few steps. The Edge Impulse Stud
147147

148148
![The Edge Impulse Studio has a built-in export function for OpenMV](assets/deployment.png)
149149

150-
Since the Nicla Vision doesn't have any on-board SRAM we need to build the machine learning model into the firmware and load it from the flash. To do so, go to https://github.com/openmv/openmv and fork the repository. In your fork click on "Actions" and enable the workflows by clicking on the green button.
150+
Since the Nicla Vision doesn't have any on-board SRAM we need to build the machine learning model into the firmware and load it from the flash. To do so, go to https://github.com/openmv/openmv and [fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the repository. In your fork click on "Actions" and enable the workflows by clicking on the green button.
151151

152152
Rename the machine learning model and the label file to fruit_detection.tflite and fruit_detection.txt respectively. In your fork, replace the built-in machine learning model under `src/lib/libtf/models` with the model you downloaded from Edge Impulse Studio. Commit the files and push the commit to the repository. It will build a new firmware automatically.
153153

@@ -157,7 +157,7 @@ You can inspect the build process under "Actions".
157157

158158
![In the actions section you can monitor the build process once it starts.](assets/github_actions.png)
159159

160-
Once the firmware has been built you can download it from the releases section that you can find in the "Code" tab. Put the board in bootloader mode and click on the connect symbol in the OpenMV IDE. In the dialog select "Load a specific firmware". Select the firmware that you just created and flash it to the board.
160+
Once the firmware has been built you can download it from the releases section that you can find in the "Code" tab. Put the board in bootloader mode and click on the connect symbol in the OpenMV IDE. In the dialog select "Load a specific firmware". Select `firmware.bin` in the folder that you just created and flash it to the board.
161161

162162
![In the release section you can find the generated firmware ready to download and install.](assets/github_releases.png)
163163

@@ -228,3 +228,9 @@ while(True):
228228
## Conclusion
229229

230230
You have learned about classification as a machine learning concept which categorizes a set of data into classes. You have also learned how supervised learning works and what quantization of a model means. Furthermore you have learned to train a custom TFLite machine learning model and deploy it to your board.
231+
232+
## Troubleshooting
233+
234+
### GitHub Workflow
235+
236+
If you’re encountering difficulties while using the GitHub workflow in your forked repository, make sure that you chose `MobileNetV2 96x96 0.1` as model type, otherwise the model will likely be too big. Also make sure you are changing the files in your own, forked repository and not in the original OpenMV repository.

0 commit comments

Comments
 (0)