Turning a Raspberry Pi 3B+ into a powerful object recognition edge server with Intel Movidius NCS2We turn the raspberry PI 3B+ into an object recognition server by deploying a MobileNet-SSD architecture on the for a fully-working solution using the Intel OpenVINO platformAlexey GyöriBlockedUnblockFollowFollowingMay 19The Intel NCS2 attached to a Raspberry Pi Model 3B+, the hardware used in this tutorialIn this part, we are going to use a readily compiled neural network in the Intel Neural Compute stick in order for it to be able to receive Base64 encoded images and turn them into bounding-box predictions.
Additionally an example for a front-end which sends camera input to the PI will be provided.
Please make sure to also check out the amazing write-up on how to deploy and test models by Mattio Varile.
A pre-trained and compiled model will be provided in an attachment.
Training and compiling the model for a custom dataset and more details on the front end will be part of another story, so stay tuned!OUTLINE0.
Preparing the Raspberry PI1.
Install the NOOBS image1.
Install the latest Intel OpenVINO software1.
Deploying object detection on the neural compute stick3.
Running the server and setting up auto-start on boot4.
Using a sample GUI to deploy our server5.
Conclusions and outlook0.
RequirementsRaspberry PI, preferrably a model 3B+ but any other model should do fine as well https://www.
org/productsIntel NCS2 https://software.
com/en-us/neural-compute-stickOptionalSome USB web camSome other computer for the front end to run on1.
Preparing the Raspberry PI1.
Install the NOOBS imageFlash the NOOBS image on a FAT32 formatted micro SD card.
org/downloads/Boot up the USB image normally, set an account password, connect to the internet, etc…Make sure to also install python3 and pip3 and wgetsudo apt-get updatesudo apt-get install python3-picamera python3-pip wget1.
Install the latest Intel OpenVINO softwareDownload the OpenVINO toolkitcd ~/Downloads && wget https://download.
tgzI recommend to follow the following guide up until (not including) the section “Build and Run Object Detection Samples”.
Install OpenVINO™ toolkit for Raspbian* OS – OpenVINO ToolkitThe OpenVINO™ toolkit was formerly known as the Intel® Computer Vision SDK.
These steps apply to 32-bit Raspbian* 9 OS…docs.
orgAfter doing everything successfully, you should see the following output when you open up a new terminal:1.
Deploying object detection on the neural compute stickWe are going to use a flask server which receives encoded images for prediction.
You can find all of the code in the following github repository https://github.
com/AlexeyGy/NCS2-servera) Firstly create a new folder named detection_server in your home folder.
mkdir ~/detection_server && cd detection_serverb) create a requirements.
txt file with the following content.
This file contains the packages that are needed.
flask is the web server and flask-cors is a wrapper which passes CORS headers (needed for cross-site scripts), more on it hereNote that OpenCV (cv2) is not part of this package list as that package is installed together with OpenVINO in step 1.
This is because OpenVINO provides their own flavor of cv2 including support for CNN architectures.
flask-corsflaskpybase64now runpip3 install -r requirements.
txtto install the packages automatically.
c) set up the server start script, create a file named RUN.
SHd) Download a pre-trained MobileNet-SSD architecture intermediate representation files which can differentiate between screws and rawl pegs.
Stay tuned for a second part of this tutorial where we will cover training.
mkdir models && cd models && wget https://github.
bin && wget https://github.
prototxt && wget https://raw.
xmlYou can see that the architecture contains three files, a labelmap which contains possible labels, a .
bin file which contains the frozen network weights and a .
xml file which contains the net topology.
More info on this can be found here.
Intel IR model.
Image courtesy of Intel https://software.
pnge) now let’s create the actual server, create the file server.
py with the following content.
More details on the individual functionalities will be provided below.
In line 12, we read the provided model files using the Intel OpenVino cv2.
readNet functionline 14 sets a preferrable target for our computation to run onlines 17- 19 contain some standard config for our flask server,line 23 uses the flask-cors wrapper to set the CORS header, more info herelines 25–29 are optional, they set a filter for all incoming data which does not consist of images in the right format of jpg or pngline 31 sets the default route for our flask server, we accept POST requests which contain an imageline 37 allows us to accept a threshold in addition to the image which we pass to the server s.
all predictions lower than the threshold are not returnedline 43 returns a JSON of the prediction resultsthe function in lines 46–50 does the actual image processing, we will get into the according util_mobilnet.
py file in a moment.
Here is a high level overview on what it does — firstly a preprocessing and scaling step is executed which is specific to the Mobilenet-SSD architecture — then the network does the inference (lines, 48–49) — finally a postprocessing step is executed including the threshold filteringf) finally, let’s create and look at the util_mobilnet.
py fileline 5 configures the dimensions which mobilnet requires, as it was trained on 300×300 square images, we set just this as our dimensionsthe read_labels function reads the labelmap file line by line to define the supported classesthe preprocessing function in line 21 handles colors and dimensions of the incoming image, the arbitrarily looking transformations are needed for mobilnet to process the image correctlyThe postprocess function in line 32 goes over all the predictions and filters out predictions below the threshold, additionally the background prediction is not returned3.
Running the server and setting up auto-start on bootTo minimize resource usage on the raspberry PI, we want to set-up a client server application where a flask server receives pictures and returns the bounding-box predictions.
I found out the best way to autostart flask is to a) add the flask start up to the .
bashrc file in the pi home folder and b) to set the system to autostart to cli with autologin.
a) As for adding the lines to the .
bashrc file, make sure to set the “Downloads” folder to the folder where you have downloaded your OpenVINO toolkitb) run the following line in a terminalsudo raspi-configyou will see the following screens where you should select the options 3 Boot options / 1 Desktop CLI / 2 Console Autologinselect the options as per screenshotsnow, after starting, you will see that the Raspberry will start the flask server automatically on port 5000, congratulations!4.
Using a sample GUI to deploy our serverThe following steps should be executed on another machine but they can be done on the Raspberry as well if you chose to run the GUI on it (will lead to lower performance).
clone the repository https://github.
com/AlexeyGy/NCS2-frontendgit clone https://github.
com/AlexeyGy/NCS2-frontendsimply run the RUN.
SH file to get a simple python server running.
You can access it via localhost:8000, you may have to give it access to your camera before you see any images though.
permission requestThe server will post webcam pictures to the address http://192.
2:5000/, if you are on another machine than your raspberry, make sure to customize that to the address of your raspberry.
The given sample GUI was from a pilot demonstrator at FIR which we used to demonstrate possible use-cases for a smart contract plattform.
More on that in a later post.
Conclusions and outlookHere is a gif of the whole system of front- and backend running together:Example detection of screws and rawl plugsThe running system is able to deliver 60fps video with about 8 fps on detections if ran on two machines.
I have found it actually to be true that too many detection fps lead to a worse user experience as the detections are updated too quickly!With the onset of edge computing, as other competitors like Google have also entered the race we are up for some exciting future use-cases for neural compute sticks.
Stay tuned for the training and front-end design part.
RessourcesWrite-up on how to deploy and test models by Mattio VarileIntel Neural compute stick official tutorialMore information on Movidius on Raspberry inc.
performance measurements by Adrian Rosebrock.