"Tensorflow + NNAPI" runs very slow on Samsung Galaxy S21

Di Wang 0
Viewed 34

Samsung Galaxy S21 is the only device runs slow when using tensor-flow lite and NNAPI together. Did anyone encounter this issue before?

Background

I used tensor-flow lite and NNAPI and built my app on different Android devices. Samsung Galaxy S21 is VERY slow (0.5 fps); All other devices (including Samsung Galaxy S20) is fast (12-20 fps).

Without using NNAPI, using GPU/CPU delegates, Samsung Galaxy S21 can run at (9-10 fps).

Implementation:

I set interpreter to use NNAPI:

TfLiteInterpreterOptionsSetUseNNAPI(m_options, true);

and then invoke interpreter using:

TfLiteInterpreterInvoke(m_interpreter);

Result:

I time stamped TfLiteInterpreterInvoke() method on different Android devices. Here are the results:

  • Samsung Galaxy S21: 860 ms;
  • Samsung Galaxy S20: 10 ms;
  • Any other android devices: 10-30 ms;

Profiled Trace

Then I used Android Profiler to trace the CPU usage in Samsung Galaxy S21:

ANeuralNetworksExecution_compute()                    --- 851 ms
    compute()                                         --- 851 ms
       ...
          [kernel.kallsysms]+ 0xffffffc---------().   --- 851 ms 
          [kernel.kallsysms]+ 0xffffffc---------().   --- 851 ms 
          [kernel.kallsysms]+ 0xffffffc---------().   --- 851 ms
          .....

It looks like the majority of the time is spent on the GPU kernels. Does this mean S21 GPU is not compatible with NNAPI and tensorflow?

nnapi tensorflow-lite
1 Answers

Could you try adb shell setprop debug.nn.vlog 1, rerun the test and share the logcat?

Miao Wang0
Related