How to add OpenCV in Flutter Plugin

For a project I'm working on, I've managed to create a flutter plugin which relies on OpenCV in order to process images coming from the camera. Please be aware that when I say "from the camera", I not mean real time frames from video capture, but pictures previously saved on device memory.

First of all, I followed the Flutter docs regarding creation of plugins here

When flutter has created the plugin folder and files, I've ended up with a structure similar to this (the "opencv" folder at first level is the root of plugin):

Notice the opencv-341.jar file which I added and contains the Java methods to control the native libraries, the OpenCVPlugin.java and opencv.dart which are the entry point to access Java methods from Flutter, and were created from Flutter.

The files name of .java and .dart files could be different and reflect the name you passed as argument during the plugin creation phase above.

The .jar file can be found in OpenCV package for JAVA as precompiled library. In same package you will find the native libraries needed at the end of this little guise.

The build.gradle file in [plugin_root]/android should reference the jar file like this

dependencies {
    implementation files('opencv/opencv-341.jar')
}

The OpencvPlugin.java should host the methods invoked from dart. In my case, to better manage different methods, I've created a code like this:

if (call.method.equals("processReceiptImage")) {
    if (!call.hasArgument("input")) {
        result.error("NOTFOUND", "File not found", null);
        return;
    }
    try {
        String input = call.argument("input");
        int maxWidth = (int) call.argument("maxWidth");
        int quality = (int) call.argument("quality");
        String output = call.argument("output");
        result.success(ProcessReceiptImage.run(input, maxWidth, quality, output));
        return;
    } catch (Exception e) {
        e.printStackTrace();
        result.error("UNKNOWN", e.getMessage(), null);
        return;
    }
}

The static method ProcessReceiptImage is contained in the relative file ProcessReceiptImage.java:

package com.yourplugin.opencv;

import org.opencv.core.Mat;
import org.opencv.core.MatOfInt;
import org.opencv.core.Size;
import org.opencv.imgcodecs.Imgcodecs;
import org.opencv.imgproc.Imgproc;

import java.util.ArrayList;


public class ProcessReceiptImage {
    static String run(String input, int maxWidth, int quality, String output) throws Exception {
        Mat src = Imgcodecs.imread(input, Imgcodecs.CV_LOAD_IMAGE_GRAYSCALE);
        Size size = src.size();

        int height = (int) (maxWidth * size.height / size.width);
        Imgproc.resize(src, src, new Size(maxWidth, height));

        ArrayList<Integer> parameters;
        parameters = new ArrayList();

        parameters.add(Imgcodecs.CV_IMWRITE_JPEG_QUALITY);
        parameters.add(quality);

        MatOfInt par = new MatOfInt();
        par.fromList(parameters);
        Imgcodecs.imwrite(output, src, par);
        return output;
    }
}

Since this point, if you run your app, it could fail due to the lack of library opencv_java3. In order to resolve this part, the only way I founded by now is to copy the native libraries into your final project like this (this is the root of your app, not the plugin):

I've included the final .so native library for each platform I want to target, so no matter what platform has the user, the code always can pick up the corret .so and correctly load OpenCV.

TIP: Since including all .so files can lead to a large APK, I suggest you to configure your build.gradle in order to split the package for abis, this way you will have a smaller apk which will be automatically served through Play Store.