Optimize Tensorflow Object Detection Model V2 Centernet Model for Evaluation

user_2958221 0
Viewed 877

I am using the tensorflow centernet_resnet50_v2_512x512_kpts_coco17_tpu-8 object detection model on a Nvidia Tesla P100 to extract bounding boxes and keypoints for detecting people in a video. Using the pre-trained from tensorflow.org, I am able to process about 16 frames per second. Is there any way I can imporve the evaluation speed for this model? Here are some ideas I have been looking into:

  • Pruning the model graph since I am only detecting 1 type of object (people)
    • Have not been successful in doing this. Changing the label_map when building the model does not seem to improve performance.
  • Hard coding the input size
    • Have not found a good way to do this.
  • Compiling the model to an optimized form using something like TensorRT
    • Initial attempts to convert to TensorRT did not have any performance improvements.
  • Batching predictions
    • It looks like the pre-trained model has the batch size hard coded to 1, and so far when I try to change this using the model_builder I see a drop in performance.
    • My GPU utilization is about ~75% so I don't know if there is much to gain here.
tensorrt object-detection-api object-detection tensorflow2.0 tensorflow
1 Answers

TensorRT should in most cases give a large increase in frames per second compared to Tensorflow.

centernet_resnet50_v2_512x512_kpts_coco17_tpu-8 can be found in the TensorFlow Model Zoo. Nvidia has released a blog post describing how to optimize models from the TensorFlow Model Zoo using Deepstream and TensorRT: https://developer.nvidia.com/blog/deploying-models-from-tensorflow-model-zoo-using-deepstream-and-triton-inference-server/

Now regarding your suggestions:

  • Pruning the model graph: Pruning the model graph can be done by converting your tensorflow model to a TF-TRT model.

  • Hardcoding the input size: Use the static mode in TF-TRT. This is the default mode and enabled by: is_dynamic_op=False

  • Compiling the model: My advise would be to convert you model to TF-TRT or first to ONNX and then to TensorRT.

  • Batching: Specifying the batch size is also covered in the NVIDIA blog post.

Lastly, for my model a big increase in performance came from using FP16 in my inference engine. (mixed precision) You could even try INT8 although then you first have to callibrate.

user_129522630
Related