The inference time here is too much.
The car will take a lot of time to make decisions which might lead to serious situations like accidents as well.
Hence, in such scenarios, we need a model that will give us real-time results.
The model should be able to detect objects and make inferences within microseconds.
Some of the commonly used algorithms for object detection include RCNN, Fast RCNN, Faster RCNN, and YOLO.
The aim of this article is not to deep dive into these techniques but to understand the SlimYOLOv3 architecture for real-time object detection.
If you wish to learn more about these techniques, check out the below tutorials: RCNN, Fast RCNN, Faster RCNN Implementation of Faster RCNN YOLO along with its implementation These techniques work really well when we do not need real-time detection.
Unfortunately, they tend to stumble and fall when faced with the prospect of real-time analysis.
Let’s look at some of the challenges you might encounter when trying to build your own real-time object detection model.
Challenges of Performing Real-Time Object Detection Real-time object detection models should be able to sense the environment, parse the scene and finally react accordingly.
The model should be able to identify what all types of objects are present in the scene.
Once the type of objects have been identified, the model should locate the position of these objects by defining a bounding box around each object.
So, there are two functions here.
First, classifying the objects in the image (image classification), and then locating the objects with a bounding box (object detection).
We can potentially face multiple challenges when we are working on a real-time problem: How do we deal with variations?.The variations might be of difference in the shape of objects, brightness level, etc.
Deploying object detection models.
This generally takes A LOT of memory and computation power, especially on machines we use on a daily basis Finally, we must also keep a balance between detection performance and real-time requirements.
Generally, if the real-time requirements are met, we see a drop in performance and vice versa.
So, balancing both these aspects is also a challenge So how can we overcome these challenges?.Well – this is where the crux of the article begins- the SlimYOLOv3 framework!.SlimYOLOv3 aims to deal with these limitations and perform real-time object detection with incredible precision.
Let’s first understand what SlimYOLOv3 is and then we will look at the architecture details to have a better understanding of the framework.
Introduction to SlimYOLOv3 Can you guess how a deep learning pipeline works?.Here’s a quick summary of a typical process: First, we design the model structure Fine-tune the hyperparameters of that model Train the model and Finally, evaluate it There are multiple components or connections in the model.
Some of these connections, after a few iterations, become redundant and hence we can remove these connections from the model.
Removing these connections is referred to as pruning.
Pruning will not significantly impact the performance of the model and the computation power will reduce significantly.
Hence, in SlimYOLOv3, pruning is performed on convolutional layers.
We will learn more about how this pruning is done in the next section of this article.
After pruning, we fine-tune the model to compensate for the degradation in the model’s performance.
A pruned model results in fewer trainable parameters and lower computation requirements in comparison to the original YOLOv3 and hence it is more convenient for real-time object detection.
Let’s now discuss the architecture of SlimYOLOv3 to get a better and clearer understanding of how this framework works underneath.
Understanding the Architecture of SlimYOLOv3 The below image illustrates how SlimYOLOv3 works: SlimYOLOv3 is the modified version of YOLOv3.
The convolutional layers of YOLOv3 are pruned to achieve a slim and faster version.
But wait – why are we using YOLOv3 in the first place?.Why not other object detection algorithms like RCNN, Faster RCNN?. Why YOLOv3?.There are basically two types (or two categories) of deep object detection models: Two-stage detectors Detectors belonging to the RCNN family fall under two-stage detectors.
The process contains two stages.
First, we extract region proposals and then classify each proposal and predict the bounding box.
These detectors generally lead to good detection accuracy but the inference time of these detectors with region proposals requires huge computation and run-time memory Single-stage detectors Detectors belonging to the YOLO series fall under single stage detectors.
It is a single-stage process.
These models utilize the predefined anchors that cover spatial position, scales, and aspect ratios across an image.
Hence, we do not need an extra branch for extracting region proposals.
Since all computations are in a single network, they are more likely to run faster than the Single-stage detectors.
YOLOv3 is also a single stage detector and currently the state-of-the-art for object detection Sparsity training The next step is the sparsity training of this YOLOv3 model: Here, we prune the YOLOv3 model using the following steps: First, we evaluate the importance of each component of the YOLOv3 model.
I will discuss the details of how to decide the importance of these components shortly Once the importance is evaluated, we remove the less important components The removed components can either be an individual neural connection or the network structures.
To define the importance of each component, we rank each neuron of the network based on their contribution.
There are multiple ways to do it: We can take the L1/L2 regularized means of neuron weights The mean activation of each neuron Number of times the output of a neuron wasn’t zero In SlimYOLOv3, the importance is calculated based on the L1 regularized means of neuron weights which are considered as the scaling factor.
The absolute value of these scaling factors is the importance of a channel.
To accelerate the convergence and improve the generalization of the YOLOv3 model, the batch normalization layer is used after every convolutional layer.
SlimYOLOv3 We then define a global threshold, let’s say ŷ, and discard any channel that has a scaling factor less than this threshold.
In this way, we prune the YOLOv3 architecture and get the SlimYOLOv3 architecture: While evaluating the scaling factor, the maxpool layers and the upsample layers of the YOLOv3 architecture have not been considered since they have nothing to do with the channel number of the layer number.
Fine-tuning We now have the SlimYOLOv3 model, so what’s next?.We fine-tune it so as to compensate for the degradation in performance and finally evaluate the fine-tuned model to determine whether the pruned model is suitable for deployment.
Sparsity training is actually quite effective in reducing the scaling factor and hence making the feature channels of convolutional layers sparse.
Training with a larger penalty factor of α = 0.
01, leads to aggressive decay of the scaling factor and the model starts to overfit.
In SlimYOLOv3, a penalty factor of α = 0.
0001 is used to perform channel pruning.
The code for SlimYOLOv3 has not been released yet.
My aim is to implement it as soon as it is out so I can update you guys here.
For now – watch this space.
End Notes We’ve covered a lot of ground in this article.
We saw the different object detection algorithms like RCNN, Fast RCNN, Faster RCNN, as well as the current state-of-the-art for object detection YOLO.
Then, we looked at the SlimYOLOv3 architecture which is the pruned version of YOLO and can be used for real-time object detection.
I’m excited to get my hands on the code for SlimYOLOv3!.I will try to implement SlimYOLOv3 and will share my learning with you guys.
If you have any questions, doubts or feedback related to this article, feel free to discuss with me in the comments section below.
You can also read this article on Analytics Vidhyas Android APP Share this:Click to share on LinkedIn (Opens in new window)Click to share on Facebook (Opens in new window)Click to share on Twitter (Opens in new window)Click to share on Pocket (Opens in new window)Click to share on Reddit (Opens in new window) Related Articles (adsbygoogle = window.
adsbygoogle || []).
push({});.. More details