Is it possible to significantly reduce the inference time of images by reducing the number of object classes?

user_13978939 0
Viewed 2374

I am using YOLOv4 to train my custom detector. Source: https://github.com/AlexeyAB/darknet

One of the issues while training is the computing power of GPU and available video RAM. What is the relationship between number of object classes and the time it takes to train the model? Also, is it possible to significantly reduce the inference time of images by reducing the number of object classes? The goal is to run inference on a Raspberry Pi or a Jetson Nano.

Any help is much appreciated. Thanks.

object-detection yolo
2 Answers

Change is number of classes doesn't have significant impact on inference time.

For example in case of Yolov4, which has got 3 Yolo layers, change in classes leads to change in filter size for conv layers preceding Yolo layers and some computation reduction within Yolo layers that's all. This is very minute compared to overall inference time as conv layers preceding Yolo layers are bottom layers with very small width and hight and also time spent on logic that depends upon number of classes within Yolo layer is very less.

Snapshot of Yolo layer

Here:

filters=(classes + 5)x3

Note that tinier version of yolov4 i.e tiny-yolov4 have got two Yolo layers only, instead of 3.

If your intent is to reduce inference time, especially on raspberry pi or a jetson nano, without losing on accuracy/mAP, do following things:

  • Quantisation: Run inference with INT8 instead of FP32. You can use this repo for this purpose. You can do this for both Jetson nano and raspberry pi.

  • Use inference library such as tkDNN, which is a Deep Neural Network library built with cuDNN and tensorRT primitives, specifically thought to work on NVIDIA Jetson Boards. You can use this for Jetson nano. Note that with TensorRT, you can use INT8 and FP16 instead of FP32 to reduce detection time.

Following techniques can be used to reduce inference time, but they come at the cost of significant drop in accuracy/mAP:

  • You can train the models with tinier versions rather than full Yolo versions.
  • Model Pruning - If you could rank the neurons in the network according to how much they contribute, you could then remove the low ranking neurons from the network, resulting in a smaller and faster network. Pruned yolov3 research paper and it's implementation. This is another pruned Yolov3 implementation.
user_66323420

I tried reducing the number of classes from 80 to 5 classes, I was aiming to detect vehicles only, on YOLOv3 and found a reduction in time. For example, using Intel Core i5-6300U CPU @ 2.40 GHz, the time was reduced by 50%, and for Nvidia GeForce 930M, it was reduced by 20%. Generally, the stronger the processor, the less reduction in time you get.

user_116529430
Related