Can we run tensorflow lite on linux ? Or it is for android and ios only

I think the other answers are quite wrong.

Look, I'll tell you my experience... I've been working with Django for many years, and I've been using normal tensorflow, but there was a problem with having 4 or 5 or more models in the same project. I don't know if you know Gunicorn + Nginx. This generates workers, so if you have 4 machine learning models, for every worker it multiplies, if you have 3 workers you will have 12 models preloaded in RAM. This is not efficient at all, because if the RAM overflows your project will fall or in fact the service responses are slower.

So this is where Tensorflow lite comes in. Switching from a tensorflow model to tensorflow lite improves and makes things much more efficient. Times are reduced absurdly. Also, Django and Gunicorn can be configured so that the model is pre-loaded and compiled at the same time. So every time the API is used up, it only generates the prediction, which helps you make each API call a fraction of a second long. Currently I have a project in production with 14 models and 9 workers, you can understand the magnitude of that in terms of RAM. And besides doing thousands of extra calculations, outside of machine learning, the API call does not take more than 2 seconds. Now, if I used normal tensorflow, it would take at least 4 or 5 seconds.

In summary, if you can use tensorflow lite, I use it daily in Windows, MacOS, and Linux, it is not necessary to use Docker at all. Just a python file and that's it. If you have any doubt you can ask me without any problem.

Here a example project Django + Tensorflow Lite

It's possible to run (but it will works slower, than original tf)

Example

# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path=graph_file)
interpreter.allocate_tensors()

# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

# Get quantization info to know input type
quantization = None
using_type = input_details[0]['dtype']
if dtype is np.uint8:
    quantization = input_details[0]['quantization']

# Get input shape
input_shape = input_details[0]['shape']

# Input tensor
input_data = np.zeros(dtype=using_type, shape=input_shape)

# Set input tensor, run and get output tensor
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])

I agree with Nouvellie. It is possible and worth the time implementing. I developed a model on my Ubuntu 18.04 32 processor server and exported the model to tflite. The model ran in 178 secs on my ubuntu server. On my raspberry pi4 with 4GB memory, the tflite implementation ran in 85 secs, less than half the time of my server. When I installed tflite on my server the run time went down to 22 secs, an 8 fold increase in performance and now almost 4 times faster than the rpi4.

To install for python, I did not have to build the package but was able to use one of the prebuilt interpreters here:

https://www.tensorflow.org/lite/guide/python

I have Ubuntu 18.04 with python 3.7.7. So I ran pip install with the Linux python 3.7 package:

pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_x86_64.whl

Then import the package with:

from tflite_runtime.interpreter import Interpreter

Previous posts show how to use tflite.

From Tensorflow lite

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices.

Tensorflow lite is a fork of tensorflow for embedded devices. For PC just use the original tensorflow.

From github tensorflow:

TensorFlow is an open source software library

TensorFlow provides stable Python API and C APIs as well as without API backwards compatibility guarantee like C++, Go, Java, JavaScript and Swift.

We support CPU and GPU packages on Linux, Mac, and Windows.

>>> import tensorflow as tf
>>> tf.enable_eager_execution()
>>> tf.add(1, 2)
3
>>> hello = tf.constant('Hello, TensorFlow!')
>>> hello.numpy()
'Hello, TensorFlow!'

Yes, you can compile Tensorflow Lite to run on Linux platforms even with a Docker container. See the demo: https://sconedocs.github.io/tensorflowlite/