Android Camera X ImageAnalyzer Image Format for TFLite

user_2023394 0
Viewed 1687

I am attempting to analyze camera preview frames with a tflite model, using the CameraX api.

This documentation describes using the ImageAnalyzer to process incoming frames. Currently the frames are incoming as YUV, and I'm not sure how to pass YUV image data to a tflite model thats expecting an input of the shape (BATCHxWIDTHxHEIGHTx3). In the old APIs you could specify preview output formats and change it to rgb, however this page specifically says "CameraX produces images in YUV_420_888 format."

First I'm hoping someone has found a way to pass RGB to the Analyzer rather than YUV, and secondly if not, could someone suggest a way of passing a YUV image to a TFLite interpreter? The incoming image object is of the type ImageProxy and it has 3 planes, Y, U, and V.

tensorflow-lite android-camerax android
1 Answers

AFAIK, the ImageAnalysis use case only provides images in the YUV_420_888 format (You can see it defined here).

The official CameraX documentation provides a way to convert YUV images to RGB bitmaps, it's at the bottom of this section.

For sample code that shows how to convert a Media.Image object from YUV_420_888 format to an RGB Bitmap object, see YuvToRgbConverter.kt.

user_47182830
Related