How to use Keras SavedModel from C++

user_8727339 0
Viewed 1609

I have a trained a classic CNN(pre-trained mobile net) for image classification. I want to now use this model from c++. From my understanding, I need to create a library of the model, that can accept the input and return its outputs. I have the model saved in format .pb (SavedModel).

I have already tried, CppFlow, where the error shows that it can't read my model. I assume it's due to incompatibility with TF 2.0.

I have also got the command line interface of SavedModel working, but I don't know how to use it in my cpp application.

I want to know how I can build a library of my model and use this library such that it can make predictions on the fly. Any guidance will be helpful. Please let me know if any additional information is required.

keras tensorflow deep-learning computer-vision c++
2 Answers

One way of using keras model in C++ is to convert it to TensorFlow .pb format. I've just composed a script for doing this, down below.

Usage: python script.py keras_model.hdf5

It outputs tensorflow model as input file name appended by .pb.

Then you can use TF C++ api for reading model and doing inference. Nice detailed example of using image recognition model to label images in C++ TF is located here.

Another option - you may use Keras directly by calling Python API from C++, it is not that difficult, there is standalone python which is compiled statically meaning having no dll/shared libs dependencies at all hence python interpreter can be fully compiled into C++ single binary. There are also many libraries in Internet that help you to easily run Python from C++.

import sys, os
from keras import backend as K
from keras.models import load_model
import tensorflow as tf

def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
    """
    Freezes the state of a session into a pruned computation graph.

    Creates a new computation graph where variable nodes are replaced by
    constants taking their current value in the session. The new graph will be
    pruned so subgraphs that are not necessary to compute the requested
    outputs are removed.
    @param session The TensorFlow session to be frozen.
    @param keep_var_names A list of variable names that should not be frozen,
                          or None to freeze all the variables in the graph.
    @param output_names Names of the relevant graph outputs.
    @param clear_devices Remove the device directives from the graph for better portability.
    @return The frozen graph definition.
    """
    from tensorflow.python.framework.graph_util import convert_variables_to_constants
    graph = session.graph
    with graph.as_default():
        freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
        output_names = output_names or []
        output_names += [v.op.name for v in tf.global_variables()]
        # Graph -> GraphDef ProtoBuf
        input_graph_def = graph.as_graph_def()
        if clear_devices:
            for node in input_graph_def.node:
                node.device = ""
        frozen_graph = convert_variables_to_constants(session, input_graph_def,
                                                      output_names, freeze_var_names)
        return frozen_graph
        
if len(sys.argv) <= 1:
    print('Usage: python script.py keras_model.hdf5')
    sys.exit(0)
else:
    ifname = sys.argv[1]

model = load_model(ifname)

frozen_graph = freeze_session(
    K.get_session(),
    output_names = [out.op.name for out in model.outputs],
)

tf.io.write_graph(frozen_graph, os.path.dirname(ifname), ifname + '.pb', as_text = False)
user_9415310
user_119979330
Related