Video file to image stream in flutter

user_13277578 0
Viewed 2350

I have a video file in my assets folder (like mp4). I want to read that video file as an image stream, like an image stream in the camera package https://pub.dev/documentation/camera/latest/camera/CameraController/startImageStream.html

I want to pass the image to Tflite model.

Something like that:

controller = loadVideo('assets/example.mp4');  // how to do that?! 
controller.startImageStream((CameraImage img) {
          Tflite.runPoseNetOnFrame(
              bytesList: img.planes.map((plane) {
                return plane.bytes;
              }).toList(),
              imageHeight: img.height,
              imageWidth: img.width,
              numResults: 1,
              threshold: 0.1,
            ).then((recognitions) {
              // ...
            }
        }

I tried to search in the image_picker and video_player packages, but did not find anything.

tensorflow dart flutter video image
2 Answers

What I did is to split the images and save them as files and then send it to the tf model. It's not fast (because you make file and then read it) but it is working, tested on android and it should work on IOS :)

I made a PR in the package flutter_export_video_frame, now it have a function in the name of exportImagesFromFile. It is Returning a Stream of images from video file as Stream<File>.

To get the video file from the asset folder I use the following function:

import 'dart:async';
import 'dart:io';
import 'package:path_provider/path_provider.dart';
import 'package:flutter/services.dart' show rootBundle;
import 'package:path/path.dart';

Future<File> getImageFileFromAssets(String path) async {
  final byteData = await rootBundle.load('assets/$path');
  final fileName = basename(path);

  final file = new File('${(await getTemporaryDirectory()).path}/$fileName');
  await file.writeAsBytes(byteData.buffer
      .asUint8List(byteData.offsetInBytes, byteData.lengthInBytes));

  return file;
}

And then to move it to the posenet/model, there is a function that gets File as input:

var results = await Tflite.runPoseNetOnImage(
      path: path,
      numResults: 1,
    );

One possible improvement to this method is to make it works only on the RAM (after you read the video, of course)

user_132775780

Yes, sure you can do that in one of the following ways:

  1. Use the export_video_frame plugin

  2. Use video_thumbnail plugin

You can specify an exact frame by the time parameter.

user_3051350
Related