How to correctly load an image for object detection in flutter?

Viewed 622

I am trying to make an object detection app with Flutter mainly using the following plugins:

"tflite_flutter" https://pub.dev/packages/tflite_flutter
"camera" https://pub.dev/packages/camera
"image_picker" https://pub.dev/packages/image_picker

When I try to detect the desired object in images loaded from the assets everything works perfectly:

import 'package:image/image.dart' as img;

// load bytes
img.Image image = img.decodeImage((await rootBundle.load('assets/images/test.JPG')).buffer.asUint8List());

// my code
List<Detection> detections = ssdMobileDet.detect(image);
image = Util.drawDetections(
  src: image,
  detections: detections,
);

// show image in app

but when I try to use the "camera" plugin:
https://flutter.dev/docs/cookbook/plugins/picture-using-camera (full example)

String path = join(
  (await getTemporaryDirectory()).path,
  '${DateTime.now()}.png',
);

await _controller.takePicture(path);

img.Image image = img.decodeImage(File(path).readAsBytesSync());

or the "image_picker" plugin:

String path = (await ImagePicker().getImage(source: ImageSource.camera)).path;
img.Image image = img.decodeImage(File(path).readAsBytesSync());

on the same image the detections are useless.

I suspect that the method rootBundle.load does something different, but I could be completely wrong.

1 Answers

I fixed my issue by adding the following line of code...

image = img.bakeOrientation(image);

This basically reads the exif data of the image and rotates it according to the tags. I knew about this "issue" but manually rotating the image (which the above method does aswell):

image = img.copyRotate(image, -90);

did not help. I am still not 100% sure why the second method does not work, i guess that removing the orientation exif tags (which the first method does) is of importance for one or more of the other operations I am applying to the image.

Related