All we need to do is the following:cd /usr/local/lib/python3.5/site-packages/sudo mv cv2.cpython-35m-arm-linux-gnueabihf.so cv2.soRun Tensorflow graphAll code below is available on GitHub.First, define an abstract class for object detectors:Next, implement the interface for TensorFlow model, we used code from the object detection API.Now let’s implement functions for video streaming from jetson onboard camera:And use it:So with SSD Lite TensorFlow model, we have around 4.2 FPS.SSD MobileNet v2 Lite ResultsRun TensorFlow Lite optimized graphTensorFlow Lite is the official solution for running machine learning models on mobile and embedded devices..It enables on‑device machine learning inference with low latency and a small binary size on Android, iOS etc..TensorFlow Lite uses many techniques for this such as quantized kernels that allow smaller and faster (fixed-point math) models.How setup environment and convert graph with TensorFlow Lite you can read in the previous article..Also, you can download a lite graph from here (detect.tflite). We need to make another detection class to work with an optimized graph..Code is available here.SSD Lite model with TensorFlow Lite optimization has 4.7 FPS on average.Run TensorRT optimized graphYou can skip this part too since we’ve made a pre-trained model available here (ssdlite_inception_v2_coco_trt.pb). Let’s consider a more detailed example.We have already downloaded a pre-trained SSD Lite model..And we should build a frozen graph:Next, build a TensorRT/Jetson compatible graph and save it:And we can use this optimized graph like a standard frozen graph.After TensorRT optimization we have 11.5 FPSTensorRT ResultsSummaryAfter our test runs we have the following performance results:Standard TensorFlow graph — 4.2 FPS;TensorFlow Lite optimized graph — 4.8 FPS;TensorRT optimized graph — 11.4 FPS.Models’ comparisonAs you can see above, TensorRT optimization almost triples out model’s performance..And if we can process images in batches, performance may grow even higher..It seems that TensorFlow Lite optimization has a small impact on Jetson, this might be due to it not being optimized for GPU use..TensorRT has only one con: not all models could be optimized with it.All in all, NVIDIA Jetson TX2 + TensorRT is a relatively inexpensive, compact and productive machine, that could be used for real-time object detection.Originally published at www.quantumobile.com on December 27, 2018.. More details