Unable to implement movenet multipose in flutter using tflite_flutter package

Viewed 45

I currently trying to implement a movenet model in Flutter using the tflite_flutter package.

I have successfully recreated the object detection example in the document with Flutter 3.0 (the sample has not been updated for a long time). But when I try to implement the Posenet model, I encounter an error.

Bad state: failed precondition
E/flutter (27000): #0      checkState (package:quiver/check.dart:74:5)
E/flutter (27000): #1      Tensor.setTo (package:tflite_flutter/src/tensor.dart:146:5)
E/flutter (27000): #2      Interpreter.runForMultipleInputs (package:tflite_flutter/src/interpreter.dart:186:33)
E/flutter (27000): #3      Interpreter.run (package:tflite_flutter/src/interpreter.dart:157:5)

Note that I have no prior experience in the AI field nor how to use Tensorflow.

Here is my implementation code

import 'dart:math';
import 'dart:ui';

import 'package:image/image.dart';
import 'package:tflite_flutter/tflite_flutter.dart';
import 'package:tflite_flutter_helper/tflite_flutter_helper.dart';
import 'package:untitled/features/pose_detection/utils/recognition.dart';

class Classifier {
  static const String modelFileName = 'pose.tflite';
  static const String labelFileName = 'labelmap.txt';
  Interpreter? interpreter;
  List<String> labels = [];
  List<List<int>> outputShapes = [];
  ImageProcessor? imageProcessor;
  TensorBuffer? outputBuffer;
  TfLiteType? inputType;

  /// Input size of image (height = width = 300)
  static const int inputSize = 300;

  /// Number of results to show
  static const int numResults = 2;

  /// Result score threshold
  static const double threshold = 0.3;

  /// Types of output tensors
  List<TfLiteType> outputTypes = [];

  Classifier({List<String>? labels, Interpreter? interpreter}) {
    loadModel(interpreter: interpreter);
  }

  void loadModel({Interpreter? interpreter}) async {
    try {
      final localInterpreter = interpreter ??
          await Interpreter.fromAsset(
            modelFileName,
            options: InterpreterOptions()..threads = 4,
          );
      final outputTensors = localInterpreter.getOutputTensors();
      this.interpreter = localInterpreter;
      outputBuffer = TensorBuffer.createFixedSize(
        outputTensors[0].shape,
        outputTensors[0].type,
      );
      inputType = localInterpreter.getInputTensor(0).type;
    } catch (e) {
      print('Error creating interpreter: $e');
    }
  }

  List<Recognition> predict(Image image) {
    if (interpreter == null || outputBuffer == null || inputType == null) {
      throw Exception('Interpreter is not loaded');
    }
    var inputImage = TensorImage(inputType!);
    inputImage.loadImage(image);
    inputImage = getProcessedImage(inputImage);
    // Documentation: https://tfhub.dev/google/lite-model/movenet/multipose/lightning/tflite/float16/1
    interpreter!.run(inputImage.buffer, outputBuffer!.getBuffer());

    return [];
  }

  TensorImage getProcessedImage(TensorImage inputImage) {
    const multiplier = 32;
    const defaultSize = 256;
    final isWidthGreater = inputImage.width > inputImage.height;

    final ratio = isWidthGreater
        ? inputImage.height / inputImage.width
        : inputImage.width / inputImage.height;

    final height = isWidthGreater ? (ratio * defaultSize) : defaultSize;
    final width = isWidthGreater ? defaultSize : (ratio * defaultSize);

    final widthMultiplier = (height / multiplier).ceil();
    final heightMultiplier = (width / multiplier).ceil();

    final finalHeight = (heightMultiplier * multiplier);
    final finalWidth = (widthMultiplier * multiplier);
    return ImageProcessorBuilder()
        .add(ResizeOp(finalHeight, finalWidth, ResizeMethod.BILINEAR))
        .add(ResizeWithCropOrPadOp(finalHeight, finalWidth))
        .build()
        .process(inputImage);
  }
}

pubspec.yml

dependencies:
  flutter:
    sdk: flutter
  flutter_localizations:
    sdk: flutter
  intl: ^0.17.0


  # The following adds the Cupertino Icons font to your application.
  # Use with the CupertinoIcons class for iOS style icons.
  cupertino_icons: ^1.0.2
  get: ^4.6.5
  rxdart: ^0.27.4
  #  provider: ^6.0.3
  flutter_riverpod: ^1.0.4
  camera: ^0.8.1+3
  image: ^3.2.0
  tflite_flutter: ^0.9.0
  tflite_flutter_helper:
    git:
      url: https://github.com/filofan1/tflite_flutter_helper.git
      ref: 783f15e5a87126159147d8ea30b98eea9207ac70
0 Answers
Related