How to specify a Java object in Xamarin that correlates with a TensorFlowLite output?

Viewed 105

I have a TensorFlowLite error as follows:

Cannot copy from a TensorFlowLite tensor (model_outputs) with shape [1, 13, 13, 35] to a Java object with shape [1, 2].

and which results from the following C# (Xamarin) code:

        var interpreter = new Xamarin.TensorFlow.Lite.Interpreter(mappedByteBuffer);

        var tensor = interpreter.GetInputTensor(0);

        var shape = tensor.Shape();

        var width = shape[1];
        var height = shape[2];

        var byteBuffer = GetPhotoAsByteBuffer(bytes, width, height);

        var sr = new StreamReader(Application.Context.Assets.Open("labels.txt"));
        var labels = sr.ReadToEnd().Split('\n').Select(s => s.Trim()).Where(s => !string.IsNullOrEmpty(s)).ToList();

        var outputLocations = new float[1][] { new float[labels.Count] };
        
        var outputs = Java.Lang.Object.FromArray(outputLocations);

        interpreter.Run(byteBuffer, outputs);

The error throws at the last line, and which I assume means that the preceding two lines are incorrectly framed:

        var outputLocations = new float[1][] { new float[labels.Count] };
        
        var outputs = Java.Lang.Object.FromArray(outputLocations);

Incidentally, labels.Count equals two!

I've used the Netron app to verify that the output shape from the .tflite model is indeed shaped [1, 13, 13, 35], hence my question, How do I specify a Java object in Xamarin that correlates with a TensorFlowLite output?

EDIT:

Interestingly, I notice that:

  • When there are 2 labels, the error is "Cannot copy from a TensorFlowLite tensor (model_outputs) with shape [1, 13, 13, 35] to a Java object with shape [1, 2]"
  • When there are 4 labels, the error is: "Cannot copy from a TensorFlowLite tensor (model_outputs) with shape [1, 13, 13, 35] to a Java object with shape [1, 4]"

Also, I don't know if it's useful to add, but tensor.ShapeSignature(); is an int[4] with values 1,416,416,3.

Here's the image pre-processing method referred to in the top (it's pretty standard stuff), which takes 416 from the ShapeSignature as width and height:

        private ByteBuffer GetPhotoAsByteBuffer(byte[] bytes, int width, int height)
        {
            var modelInputSize = FloatSize * height * width * PixelSize;

            var bitmap = BitmapFactory.DecodeByteArray(bytes, 0, bytes.Length);
            var resizedBitmap = Bitmap.CreateScaledBitmap(bitmap, width, height, true);

            var byteBuffer = ByteBuffer.AllocateDirect(modelInputSize);
            byteBuffer.Order(ByteOrder.NativeOrder());

            var pixels = new int[width * height];
            resizedBitmap.GetPixels(pixels, 0, resizedBitmap.Width, 0, 0, resizedBitmap.Width, resizedBitmap.Height);

            var pixel = 0;

            for (var i = 0; i < width; i++)
            {
                for (var j = 0; j < height; j++)
                {
                    var pixelVal = pixels[pixel++];

                    byteBuffer.PutFloat(pixelVal >> 16 & 0xFF);
                    byteBuffer.PutFloat(pixelVal >> 8 & 0xFF);
                    byteBuffer.PutFloat(pixelVal & 0xFF);
                }
            }

            bitmap.Recycle();

            return byteBuffer;
        }
0 Answers
Related