Quantized models in Object Detection API for TF2

Question

I want to migrate my code for fine-tuning an object detection model for inference on Coral devices to TensorFlow 2, but I don't see quantized models in the TF2 model zoo.

Is it possible to fine-tune a model in TF2 for this purpose and use a technique like quantization-aware training or post-training quantization? I haven't seen any related tutorials or issues. I've also seen some reports of issues with quantization with TFLite converter in TF2 so I'm not even sure if it's possible to do it in TF2.

user_5399778 · Answer

Even for the TF1.x models, the conversions are not that straightforward which I am struggling with them at this time. I am pretty disappointed as I think Coral should have better support for tensorflow than Intel NCS USB originally as they are in the same family. However, it seems that I am wrong.

user_11294638 · Answer

Quantization aware training is possible by adding graph_rewriter at the end of the config file before fine-tuning of the pretrained model:

graph_rewriter {
  quantization { 
  delay: 48000
  weight_bits: 8
  activation_bits: 8
  } 
}

Source: https://neuralet.com/article/quantization-of-tensorflow-object-detection-api-models/

Quantized models in Object Detection API for TF2

2 Answers