Tensorflow lite object detection, Android Studio, ssd mobilevet v2, same structure different tflite file but almost 0 detection

user_18788203 0
Viewed 230

I want to make object detection application base on this github https://github.com/bendahouwael/Vehicle-Detection-App-Android.

That github code uses tflite based on ssd mobilenet v1. So I made my custom model based on ssd mobilenet v2. I followed this link https://colab.research.google.com/drive/1qXn9q6m5ug7EWJsJov6mHaotHhCUY-wG?usp=sharing to make my own TFLITE model.

From https://netron.app/ I checked the model structure both almost same. Please see the pictures below. This structure is about SSD MOBILENET V1

First picture is about SSD MOBILENET V1 Structure.

enter image description here

Second picture is about my own custom model based on SSD MOBILENET V2.

I think both models' structure is same. So I just pasted my own model into app code(to asset folder) with label txt file.

The application showed its real time image well but did not detect the objects that I decided what to detect. I know ssd mobilenet V1 type is unit8 and my own model (which is based on ssd mobilenet v2) type is float32. But this is not a problem I guess b/c in the code it has setting about quantized or not.

So please who has any ideas, tell me the reason why my application works so bad.

ps1) I forgot to say about debugging. It did not show any error messages. This makes me much hard to work

mobilenet object-detection tensorflow-lite tensorflow android
1 Answers

If you look closely at INPUT part,

  • with MobileNet V1 you have: type: unit8[1, 300, 300, 1]
  • with MobileNet V2 you have: type: float[1, 300, 300, 1]

This means that the first model is quantized (more info: here) and for the weight and biases use integer values. (this is done for inference speed)

Now if you go to your TFlite Object Detection class (or maybe named different), usually you will have a method called recognizeImage() similar like this (this is the part when you create fill the ByteBuffer):

imgData.rewind();
for (int i = 0; i < inputSize; ++i) {
  for (int j = 0; j < inputSize; ++j) {
    int pixelValue = intValues[i * inputSize + j];
    if (isModelQuantized) {
      // Quantized model
      imgData.put((byte) ((pixelValue >> 16) & 0xFF));
      imgData.put((byte) ((pixelValue >> 8) & 0xFF));
      imgData.put((byte) (pixelValue & 0xFF));
    } else { // Float model
      imgData.putFloat((((pixelValue >> 16) & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
      imgData.putFloat((((pixelValue >> 8) & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
      imgData.putFloat(((pixelValue & 0xFF) - IMAGE_MEAN) / IMAGE_STD);
    }
  }
}

where:

  private static final float IMAGE_MEAN = 128.0f;
  private static final float IMAGE_STD = 128.0f;

So in the first case set the isModelQuantized = true, and for MobileNet V2 you set isModelQuantized = false

user_135661280
Related