Running tflite sample segmentation app with different model

user_5585247 0
Viewed 379

I am trying to run the sample app from tensorflow for image segmentation with a different model. I would like to run it with the model shufflenetv2 with dpc. So I copied the model, and changed imageSize to 225 in ImageSegmentationModelExecutor.kt. Then I am getting the error

something went wrong: y + height must be <= bitmap.height()

Doing some small adjustments in the function scaleBitmapAndKeepRatio of ImageUtils.kt solves the problem. (Just changed targetBmp.width to height twice, once in the matrix and 2nd time in the return.) This brings the next error

something went wrong: Cannot convert between a TensorFlowLite buffer with 202500 bytes and a Java Buffer with 4252500 bytes.

The ratio of these 2 numbers is the NUM_CLASSES. Not sure if this is the right way to get it running or how to continue from here. Any ideas or suggestions?

tensorflow-lite tensorflow android
1 Answers

You seem to have two unrelated problems

1) The method scaleBitmapAndKeepRatio seems buggy.

By replacing width with height you're not solving the problem. Only changing the moment when it will happen. See the reference for createBitmap

if the x, y, width, height values are outside of the dimensions of the source bitmap, or width is <= 0, or height is <= 0, or if the source bitmap has already been recycled

so to get a squared adjusted bitmap, you'd be better off by getting the smallest dimension like this

val dimension = min(targetBmp.height, targetBmp.width)

and replacing width with dimension

2) I believe the input node of your tflite model is not compatible with tflite segmentation example

Have a look to the default model Google provides with Netron

You can see the the default model input node is a quantized float32 To me, it seems likely that you've converted your model to tflite using the default instructions in quantize.md. This will get you a model expecting a quantized uint8 input node, and so, the datatype is a mismatch. Remember tflite examples in the repository are taylor-made for an specific input and not very generic.

You'd rather do the conversion shown below to get as input node a quantized float32

# Load the TensorFlow model
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(
    graph_def_file = MODEL_FILE, 
    input_arrays = ['sub_2'], # For the Xception model it needs to be `sub_7`, for MobileNet it would be `sub_2`
    output_arrays = ['ResizeBilinear_2'], # For the Xception model it needs to be `ResizeBilinear_3`, for MobileNet it would be `ResizeBilinear_2`
    input_shapes={'sub_2':[1,257,257,3]}
)
user_30371070
Related