I'm trying to inference my tflite model on c++ code at an embedded device.
So, I type simple tflite inference code which is using GPU.
And I cross-compile in my PC and run at the embedded device which is running android.
However,
(1) If I use delegate gpu option, c++ codes give random results.
(2) It has given same input, but the results changed every time.
(3) When I turn off the gpu option, it gives me a correct results.
When I test my tflite model in python, it gives the correct output.
So I think the model file has no problem.
Should I re-build TensorFlow-lite because I use a prebuilt .so file? or Should I change the GPU option?
I don't know what should I check more.
Please Help!
Here is my C++ code
// Load model
std::unique_ptr<tflite::FlatBufferModel> model =
tflite::FlatBufferModel::BuildFromFile(model_file.c_str());
// Build the interpreter
tflite::ops::builtin::BuiltinOpResolver resolver;
std::unique_ptr<tflite::Interpreter> interpreter;
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
// set delegate option
bool use_gpu = true;
if(use_gpu)
{
TfLiteDelegate* delegate;
auto options = TfLiteGpuDelegateOptionsV2Default();
options.inference_preference = TFLITE_GPU_INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER;
options.inference_priority1 = TFLITE_GPU_INFERENCE_PRIORITY_AUTO;
delegate = TfLiteGpuDelegateV2Create(&options);
interpreter->ModifyGraphWithDelegate(delegate);
}
interpreter->AllocateTensors();
// set input
float* input = interpreter->typed_input_tensor<float>(0);
for(int i=0; i<width*height*channel; i++)
*(input+i) = 1;
TfLiteTensor* output_tensor = nullptr;
// Inference
interpreter->Invoke();
// Check output
output_tensor = interpreter->tensor(interpreter->outputs()[0]);
printf("Result : %f\n",output_tensor->data.f[0]);
//float* output = interpreter->typed_output_tensor<float>(0);
//printf("output : %f\n",*(output));