How to convert frozen graph to TensorFlow lite

user_416206 0
Viewed 605

I have been trying to follow, https://www.tensorflow.org/lite/examples/object_detection/overview#model_customization all day to convert any of the tensorflow Zoo models to a TensorFlow Lite model for running on Android with no luck.

I downloaded several of the models from here, https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md (FYI, Chrome does not let you down these links as not https, I had to right-click Inspect the link and click on the link in the inspector)

I have the script,

import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_frozen_graph(
    graph_def_file='frozen_graph.pb',
    input_shapes = {'normalized_input_image_tensor':[1,300,300,3]},
    input_arrays = ['normalized_input_image_tensor'],
    output_arrays = ['TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1', 'TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3']
)
tflite_model = converter.convert()

with open('model.tflite', 'wb') as f:
  f.write(tflite_model)

but gives the error, ValueError: Invalid tensors 'normalized_input_image_tensor' were found

so the lines,

input_shapes = {'normalized_input_image_tensor':[1,300,300,3]}, input_arrays = ['normalized_input_image_tensor'], output_arrays = ['TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1', 'TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3']

must be wrong, need a different shape, but how do I get this for each of the zoo models, or is there some preconvert code I need to run first?

Running the "code snipet" below I get,

--------------------------------------------------
Frozen model layers:
name: "add/y"
op: "Const"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "value"
  value {
    tensor {
      dtype: DT_FLOAT
      tensor_shape {
      }
      float_val: 1.0
    }
  }
}

Input layer:  add/y
Output layer:  Postprocessor/BatchMultiClassNonMaxSuppression/map/while/NextIteration_1
--------------------------------------------------

But I don't see how this would map to the input_shape or help with the conversion??

Is it even possible to convert models like faster_rcnn_inception_v2_coco to tflite? I read somewhere that only SSD models are supported?

So I tried to convert the faster_rcnn_inception_v2_coco to tflite using the below suggested code, the conversation code did not work in TF1, but did work in TF2, but when I try to use the tflite file in the TFlite Example app I get this error,

2021-12-14 13:23:01.979 24542-24542/org.tensorflow.lite.examples.detection E/tflite: Missing 'operators' section in subgraph.
2021-12-14 13:23:01.984 24542-24542/org.tensorflow.lite.examples.detection E/TaskJniUtils: Error getting native address of native library: task_vision_jni
    java.lang.RuntimeException: Error occurred when initializing ObjectDetector: Could not build model from the provided pre-loaded flatbuffer: Missing 'operators' section in subgraph.
        at org.tensorflow.lite.task.vision.detector.ObjectDetector.initJniWithByteBuffer(Native Method)
        at org.tensorflow.lite.task.vision.detector.ObjectDetector.access$100(ObjectDetector.java:88)
        at org.tensorflow.lite.task.vision.detector.ObjectDetector$3.createHandle(ObjectDetector.java:223)
        at org.tensorflow.lite.task.core.TaskJniUtils.createHandleFromLibrary(TaskJniUtils.java:91)
        at org.tensorflow.lite.task.vision.detector.ObjectDetector.createFromBufferAndOptions(ObjectDetector.java:219)
        at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.<init>(TFLiteObjectDetectionAPIModel.java:88)
        at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.create(TFLiteObjectDetectionAPIModel.java:82)
        at org.tensorflow.lite.examples.detection.DetectorActivity.onPreviewSizeChosen(DetectorActivity.java:99)
        at org.tensorflow.lite.examples.detection.CameraActivity$7.onPreviewSizeChosen(CameraActivity.java:446)
tensorflow-lite tensorflow android python
2 Answers

This code snippet

import tensorflow as tf

def print_layers(graph_def):
    def _imports_graph_def():
        tf.compat.v1.import_graph_def(graph_def, name="")

    wrapped_import = tf.compat.v1.wrap_function(_imports_graph_def, [])
    import_graph = wrapped_import.graph

    print("-" * 50)
    print("Frozen model layers: ")
    layers = [op.name for op in import_graph.get_operations()]
    ops = import_graph.get_operations()
    print(ops[0])
    print("Input layer: ", layers[0])
    print("Output layer: ", layers[-1])
    print("-" * 50)

# Load frozen graph using TensorFlow 1.x functions
with tf.io.gfile.GFile("model.pb", "rb") as f:
    graph_def = tf.compat.v1.GraphDef()
    loaded = graph_def.ParseFromString(f.read())

frozen_func = print_layers(graph_def=graph_def)

prints the attributes, including the shape, of the input layer, along with the names of input and output layers:

--------------------------------------------------
Frozen model layers: 
name: "image_tensor"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_UINT8
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: -1
      }
      dim {
        size: -1
      }
      dim {
        size: -1
      }
      dim {
        size: 3
      }
    }
  }
}

Input layer:  image_tensor
Output layer:  detection_classes
--------------------------------------------------

You can then insert correct layer names and shape to your code, and the conversion should work.

user_116648000

Those models were made using TensorFlow version 1. so you have to use the saved_model to generate a concrete function (because TFLite doesn't like dynamic input shapes), and from there convert to TFLite.

I will write down a simple solution that you can use immediately.

Open a colab notebook, it is free and online. Go to this address and click on New Notebook at right down.

First cell (input below and execute with play button):

!wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz
!tar -xzvf "/content/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz" -C "/content/"

Second cell (input,execute):

import tensorflow as tf
print(tf.__version__)

Third cell (input, execute):

model = tf.saved_model.load('/content/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/saved_model')
concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 300, 300, 3])

converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]

tflite_model = converter.convert()

with open('detect.tflite', 'wb') as f:
  f.write(tflite_model)

The code below is necessary because there are some ops that are not supported natively by TFLite:

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]

but you have to add the specific dependency also at the mobile project following this.

If you want to shed some MB of the tflite file and make it smaller follow these procedures.

After completion you will see at the left side a detect.tflite model.

Go to netron.app and copy paste the file or browse to upload it. You will see all the details: enter image description here

user_71180840
Related