How to process image in android for tensorflow lite input?

user_11614890 0
Viewed 2781

What I Did?

I am working on image classification problem. I have trained my CNN model in python and then converted it into Tensorflow-Lite for my android app. I have tested my tensorflow-lite model in python and compared my results with my keras model results. Both are same which mean my conversion to Tensorflow lite is correct.

In python my code for reading image is as below. I am not doing any normalization or any other operation on this image. just resizing it and predicting it.

image_array_abnormal = np.array([resize(imread('/content/path/abnormal2.png'), (137, 310, 3))])

The image_array_shape is:

(1, 137, 310, 3)

Below are my input and output details of tensorflow lite:

enter image description here

Problem

But when I try to use my model in android, it does not give expected accuracy. I think there is problem with my image or input data to tensorflow lite model. In android I am just reading image from drawable, using opencv, storing it in Mat, converting it into bitmap and passing it in tensorflow for prediction as below:

Mat src = Utils.loadResource(this, R.drawable.abnormal2);
Bitmap dst = Bitmap.createBitmap(src.width(), src.height(), Bitmap.Config.ARGB_8888);

classifier = new Classifier(activity, "model.tflite", "labels.txt");
results = classifier.recognizeImage(dst);

For Classifier class i am following Tensorflow android demo classifier class.

What Help I need?

  1. Now How should I process my image that i should became same input as in python?
  2. Do I have to convert it into array as in python?
  3. Do I have to reshape it to (1, 137, 310, 3)?
  4. Do I have to create 4D array?
  5. How I will get pixel values of images?
  6. Do I have to extract RGB pixel values and then change it into 3D array or something?

I am very confused. I know I am doing mistake in image input but what is the right way? How actually I have to process my image to use it as an input to my model?

tensorflow-lite machine-learning android python java
1 Answers

It looks like you train the image classification yourself in Python. If you have full access to the training dataset, I'd recommend using TF Lite Model Maker and ML Model Binding to train and integrate the TF Lite model to your Android app. I'll be much easier than the current path. Check out this video. https://www.youtube.com/watch?v=s_XOVkjXQbU&t=540s

Back to your question, there are several reasons that can cause the issues:

  1. Incorrect normalization. It's not clear from model if it's a float or a quantized model, and if it requires normalizing to [-1, 1] or [0, 1] or [0, 255]. You may need to update the parameters on this class to make it work.

  2. Image ratio issue: Your image is very wide but most image classification model are trained with square image, so accuracy can drop as the image is resized.

user_38938600
Related