Context
I've been playing with the Google Coral USB Accelerator for a few days, prototyping some solutions based on classic models like MobileNet (official versions from here) and other custom models (ported to TensorFlow Lite). Also, I'm working specific with Python version of this libraries.
My purpose is to port a solution that I currently have running on a traditional way (CPU + GPU) to a hybrid CPU + Google Coral way, delegating the inferences to my Coral USB stick.
As my solution involves multiple models and concurrency on demand for inferences, I am using Tensorflow Serving to serve the models in the original deploy. In this way I delegate concurrency handling to TFS.
On the other hand, I have noticed that my device does not seem to support concurrency and apparently mutual exclusion is not covered by the tflite_runtime.interpreter class, which I use to make the inference.
My Question
Is there an official API at the level of TensorflowLite, Google Coral framework or some community project that can solve my problem, similar to what TFS does?
Otherwise I guess I will be forced to implement some client/server logic based for example in this open source project which uses python multiprocessing.Manager objects (see here).
Thanks a lot in advance!