TFlite with delegate gpu is give wrong result

Viewed 432

I'm trying to inference my tflite model on c++ code at an embedded device.
So, I type simple tflite inference code which is using GPU.
And I cross-compile in my PC and run at the embedded device which is running android.

However,
(1) If I use delegate gpu option, c++ codes give random results.
(2) It has given same input, but the results changed every time.
(3) When I turn off the gpu option, it gives me a correct results.

When I test my tflite model in python, it gives the correct output.
So I think the model file has no problem.

Should I re-build TensorFlow-lite because I use a prebuilt .so file? or Should I change the GPU option?
I don't know what should I check more.
Please Help!

Here is my C++ code

// Load model
std::unique_ptr<tflite::FlatBufferModel> model = 
tflite::FlatBufferModel::BuildFromFile(model_file.c_str());

// Build the interpreter
tflite::ops::builtin::BuiltinOpResolver resolver;
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);

// set delegate option
bool use_gpu = true;
if(use_gpu)
{
  TfLiteDelegate* delegate;
  auto options = TfLiteGpuDelegateOptionsV2Default();
  options.inference_preference = TFLITE_GPU_INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER; 
  options.inference_priority1 = TFLITE_GPU_INFERENCE_PRIORITY_AUTO;
  delegate = TfLiteGpuDelegateV2Create(&options);
  interpreter->ModifyGraphWithDelegate(delegate);
}

interpreter->AllocateTensors();

// set input
float* input = interpreter->typed_input_tensor<float>(0);
for(int i=0; i<width*height*channel; i++)
  *(input+i) = 1;

TfLiteTensor* output_tensor = nullptr;
    
// Inference
interpreter->Invoke();

// Check output
output_tensor = interpreter->tensor(interpreter->outputs()[0]);
printf("Result : %f\n",output_tensor->data.f[0]);
    
//float* output = interpreter->typed_output_tensor<float>(0);
//printf("output : %f\n",*(output));    
2 Answers

Luckily, I found the answer. This was caused by the mismatch of the NDK version. (The prebuilt so file I used is built another version) After unifying the NDK version to 21, it was tested again, and it worked normally.

In my case same application using gpu on S10 lite was not working but on S22,S21 Ultra 5G was working. So you can consider running on different chipset configuration on different devices with same application.

Related