Tensorflow lite on Android, How can I define the input and output of 'runForMultipleInputsOutputs' function?

Viewed 1271

I am using the tensorflow lite at the android. But, runForMulipleInputsOutputs function is not working.

this is what I did.

1. make a 'tfile', here is the source of model at the Colab

from numpy import mean
from numpy import std
from numpy import dstack
from pandas import read_csv
from matplotlib import pyplot
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Dropout
from keras.layers.convolutional import Conv1D
from keras.layers.convolutional import MaxPooling1D
from keras.utils import to_categorical
from tensorflow import keras

#make the model
n_timesteps, n_features, n_outputs = 128, 9, 6
model = Sequential()
model.add(Conv1D(filters=64, kernel_size=3, activation='relu', input_shape=(n_timesteps,n_features)))
model.add(Conv1D(filters=64, kernel_size=3, activation='relu'))
model.add(Dropout(0.5))
model.add(MaxPooling1D(pool_size=2))
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dense(n_outputs, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

#save the model
model.save("/content/gdrive/My Drive/Train_data/accel_trained_model.h5")
model2 = keras.models.load_model("/content/gdrive/My Drive/Train_data/accel_trained_model.h5")
model2.save('/content/gdrive/My Drive/Train_data/tf_accel_trained_model', save_format="tf")

#convert the model and save the tfile
converter = tf.lite.TFLiteConverter.from_saved_model('/content/gdrive/My Drive/Train_data/tf_accel_trained_model')
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
open('/content/gdrive/My Drive/Train_data/converted_model.tflite', 'wb').write(tflite_model)

2. add the tensorflow lite option at the 'build.gradle(Module)' of the Android

aaptOptions {
        noCompress "tflite"
        noCompress "lite"
    }

dependencies {
    implementation 'org.tensorflow:tensorflow-lite:+'
}

3. upload the model at the android

tflite = getTfliteInterpreter(modelFile);


private Interpreter getTfliteInterpreter(String modelPath) {
    try {
        return new Interpreter(loadModelFile(MainActivity.this, modelPath));
    }
    catch (Exception e) {
        e.printStackTrace();
    }
    return null;
}


private MappedByteBuffer loadModelFile(Activity activity, String MODEL_FILE) throws IOException {
    AssetFileDescriptor fileDescriptor = activity.getAssets().openFd(MODEL_FILE);
    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
    FileChannel fileChannel = inputStream.getChannel();
    long startOffset = fileDescriptor.getStartOffset();
    long declaredLength = fileDescriptor.getDeclaredLength();
    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}

3. make input and output, model.runForMultipleInputsOutputs

float[][] inp=new float[128][9];
float[][] out=new float[][]{{0, 0, 0, 0, 0, 0}};

java.util.Map<Integer, Object> outputs = new java.util.HashMap();
outputs.put(0, out);

tflite.runForMultipleInputsOutputs(inp,outputs);

Result) Error, I don't know what is the right input and output at the model.runForMultipleInputsOutputs

2020-03-19 22:00:45.219 14799-14799/com.example.tensorflowlite E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.tensorflowlite, PID: 14799
    java.lang.NullPointerException: Attempt to invoke virtual method 'void org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(java.lang.Object[], java.util.Map)' on a null     object reference
        at com.example.tensorflowlite.MainActivity$1.onClick(MainActivity.java:93)
        at android.view.View.performClick(View.java:6597)
        at android.view.View.performClickInternal(View.java:6574)
        at android.view.View.access$3100(View.java:778)
        at android.view.View$PerformClick.run(View.java:25885)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
1 Answers

I find what is the problem.

Frist. Keras model should not be changed to tensorflow model. Convert directly the keras model to tensorflow lite model(tfile). here is the code( save and convert model)

import tensorflow as tf
from tensorflow import keras
model2 = keras.models.load_model("/content/gdrive/My Drive/Train_data/accel_trained_model.h5")
converter = tf.lite.TFLiteConverter.from_keras_model_file("/content/gdrive/My Drive/Train_data/accel_trained_model.h5")
tflite_model = converter.convert()
open('/content/gdrive/My Drive/Train_data/converted_model.tflite', 'wb').write(tflite_model)

Second. I change the input at the android. you can check the type of input and output at the android. By doing this,

Log.d("Tag", Arrays.toString(input.shape()));
Log.d("Tag", Arrays.toString(outi.shape()));

In this case my tpye of input and output is this.

#input shape Log
2020-03-20 21:33:59.608 20035-20035/com.example.tensorflowlite D/Tag: [1, 128, 9]
#output shape Log
2020-03-20 21:33:59.608 20035-20035/com.example.tensorflowlite D/Tag: [1, 6]

So, I change the input and output shape. like this.

float[][][] inp=new float[1][128][9];
float[][] out=new float[][]{{0, 0, 0, 0, 0, 0}};

tflite = getTfliteInterpreter(modelFile);
tfile.run(inp, out);

private Interpreter getTfliteInterpreter(String modelPath) {
    try {
        return new Interpreter(loadModelFile(MainActivity.this, modelPath));
    }
    catch (Exception e) {
        e.printStackTrace();
    }
    return null;
}


private MappedByteBuffer loadModelFile(Activity activity, String MODEL_FILE) throws IOException {
    AssetFileDescriptor fileDescriptor = activity.getAssets().openFd(MODEL_FILE);
    FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
    FileChannel fileChannel = inputStream.getChannel();
    long startOffset = fileDescriptor.getStartOffset();
    long declaredLength = fileDescriptor.getDeclaredLength();
    return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
}

and then, That's work well.

Related