Run Tensorflow 2 prediction on AWS Lambda

Question

I want to run a Tensorflow 2 LinearClassifier on AWS Lambda. I have a trained LinearClassifier tensorflow model. I want to run only the prediction in AWS Lambda, because of cost efficeny.

Currently I'm not sure how to handle it.

I tried to use normal TF2, but it is too big for AWS Lambda Layers. All descriptions you find are with older TF 1.1x versions and Python 2.x.
Then I tried TensorflowJS, but it looks like the LinearClassifier using functions which are not implemented in TFJS.
Last I tried to user Tensorflow Lite, but there I currently have problems to convert my model to TFLite.

Do you have an idea, how I can get TF 2 Estimation of LinearClassifier running on AWS Lambda?

Normally I think not all of the 1.5 GB Tensorflow modules are used for the simple prediction of an existing model. So is there a possibility to only get the files which are used for running the script?

user_4201275 · Answer

I have found possible answers for your question :

To transform in TFlite model :

a blog about "How to convert a Tensorflow Estimator to Tensorflow Lite" : https://medium.com/datadriveninvestor/how-to-convert-a-tensorflow-estimator-to-tensorflow-lite-a3509a9ba902
an other example with Keras model but maybe the same trick with get_concrete_function (it works for me with my keras model): https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/experimental_new_converter/Keras_LSTM_fusion_Codelab.ipynb

To use with AWS Lambda, there are a lot of blogs but none of them is perfect. For my case, I used tensorflow 2.3.0 to create my keras model. So I have to modified a little this very good tutorial :

https://www.edeltech.ch/tensorflow/machine-learning/serverless/2020/07/11/how-to-deploy-a-tensorflow-lite-model-on-aws-lambda.html or his github : https://github.com/edeltech/tensorflow-lite-on-aws-lambda

For the part : "Clone TensorFlow from the official GitHub repository", we need to force a version of tensorflow because sometimes latest version is not compiling well.

For my case, I want tensorflow v2.3.0, so i modified this line :

git clone https://github.com/tensorflow/tensorflow.git

by

git clone --branch v.2.3.0 https://github.com/tensorflow/tensorflow.git

I have modified also at the end, to invoke lambda on AWS :

sls invoke local -f predict

by

sls invoke -f predict

I recommend you to do all the tutorial with the sample model provided to be sure everything is working.

Run Tensorflow 2 prediction on AWS Lambda

1 Answers