How to export Keras .h5 to tensorflow .pb?

Question

I have fine-tuned inception model with a new dataset and saved it as ".h5" model in Keras. now my goal is to run my model on android Tensorflow which accepts ".pb" extension only. question is that is there any library in Keras or tensorflow to do this conversion? I have seen this post so far : https://blog.keras.io/keras-as-a-simplified-interface-to-tensorflow-tutorial.html but can't figure out yet.

user_309798 · Answer

Update for Tensorflow 2

Saving everything into a single archive in the TensorFlow SavedModel format (contains saved_model.pb file):

model = ...  # Get model (Sequential, Functional Model, or Model subclass)
model.save('path/to/location')

or in the older Keras H5 format:

model = ...  # Get model (Sequential, Functional Model, or Model subclass)
model.save('model.h5')

The recommended format is SavedModel.

Loading the model back:

from tensorflow import keras
model = keras.models.load_model('path/to/location')
model = keras.models.load_model('model.h5')

A SavedModel contains a complete TensorFlow program, including trained parameters (i.e, tf.Variables) and computation. It does not require the original model building code to run, which makes it useful for sharing or deploying with TFLite, TensorFlow.js, TensorFlow Serving, or TensorFlow Hub.

Save and load Keras models: https://www.tensorflow.org/guide/keras/save_and_serialize
Using the SavedModel format: https://www.tensorflow.org/guide/saved_model

Example for Tensorflow 2

The following simple example (XOR example) shows how to export Keras models (in both h5 format and pb format), and using the model in Python and C++:

train.py:

import numpy as np
import tensorflow as tf

print(tf.__version__)  # 2.4.1

x_train = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], 'float32')
y_train = np.array([[0], [1], [1], [0]], 'float32')

inputs = tf.keras.Input(shape=(2,), name='input')
x = tf.keras.layers.Dense(64, activation='relu')(inputs)
x = tf.keras.layers.Dense(64, activation='relu')(x)
x = tf.keras.layers.Dense(64, activation='relu')(x)
x = tf.keras.layers.Dense(64, activation="relu")(x)
outputs = tf.keras.layers.Dense(1, activation='sigmoid', name='output')(x)

model = tf.keras.Model(inputs=inputs, outputs=outputs, name='xor')

model.summary()

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['binary_accuracy'])

model.fit(x_train, y_train, epochs=100)

model.save('./xor/')  # SavedModel format

model.save('./xor.h5')  # Keras H5 format

After run the above script:

.
├── train.py
├── xor
│   ├── assets
│   ├── saved_model.pb
│   └── variables
│       ├── variables.data-00000-of-00001
│       └── variables.index
└── xor.h5

predict.py:

import numpy as np
import tensorflow as tf

print(tf.__version__)  # 2.4.1

model = tf.keras.models.load_model('./xor/')  # SavedModel format
# model = tf.keras.models.load_model('./xor.h5')  # Keras H5 format

# 0 xor 0 = [[0.11921611]] ~= 0
print('0 xor 0 = ', model.predict(np.array([[0, 0]])))

# 0 xor 1 = [[0.96736085]] ~= 1
print('0 xor 1 = ', model.predict(np.array([[0, 1]])))

# 1 xor 0 = [[0.97254556]] ~= 1
print('1 xor 0 = ', model.predict(np.array([[1, 0]])))

# 1 xor 1 = [[0.0206149]] ~= 0
print('1 xor 1 = ', model.predict(np.array([[1, 1]])))

Convert Model to ONNX:

ONNX is a new standard for exchanging deep learning models. It promises to make deep learning models portable thus preventing vendor lock in.

ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.

$ pip install onnxruntime
$ pip install tf2onnx
$ python -m tf2onnx.convert --saved-model ./xor/ --opset 9 --output xor.onnx

# INFO - Successfully converted TensorFlow model ./xor/ to ONNX
# INFO - Model inputs: ['input:0']
# INFO - Model outputs: ['output']
# INFO - ONNX model is saved at xor.onnx

By specifying --opset the user can override the default to generate a graph with the desired opset. For example --opset 13 would create a onnx graph that uses only ops available in opset 13. Because older opsets have in most cases fewer ops, some models might not convert on a older opset.

opencv-predict.py:

import numpy as np
import cv2

print(cv2.__version__)  # 4.5.1

model = cv2.dnn.readNetFromONNX('./xor.onnx')

# 0 xor 0 = [[0.11921611]] ~= 0
model.setInput(np.array([[0, 0]]), name='input:0')
print('0 xor 0 = ', model.forward(outputName='output'))

# 0 xor 1 = [[0.96736085]] ~= 1
model.setInput(np.array([[0, 1]]), name='input:0')
print('0 xor 1 = ', model.forward(outputName='output'))

# 1 xor 0 = [[0.97254556]] ~= 1
model.setInput(np.array([[1, 0]]), name='input:0')
print('1 xor 0 = ', model.forward(outputName='output'))

# 1 xor 1 = [[0.02061491]] ~= 0
model.setInput(np.array([[1, 1]]), name='input:0')
print('1 xor 1 = ', model.forward(outputName='output'))

predict.cpp:

#include 
#include 
#include 

int main(int argc, char **argv)
{
    std::cout << CV_VERSION << std::endl; // 4.2.0

    cv::dnn::Net net;

    net = cv::dnn::readNetFromONNX("./xor.onnx");

    // 0 xor 0 = [0.11921611] ~= 0
    float x0[] = { 0, 0 };
    net.setInput(cv::Mat(1, 2, CV_32F, x0), "input:0");
    std::cout << "0 xor 0 = " << net.forward("output") << std::endl;

    // 0 xor 1 = [0.96736085] ~= 1
    float x1[] = { 0, 1 };
    net.setInput(cv::Mat(1, 2, CV_32F, x1), "input:0");
    std::cout << "0 xor 1 = " << net.forward("output") << std::endl;

    // 1 xor 0 = [0.97254556] ~= 1
    float x2[] = { 1, 0 };
    net.setInput(cv::Mat(1, 2, CV_32F, x2), "input:0");
    std::cout << "1 xor 0 = " << net.forward("output") << std::endl;

    // 1 xor 1 = [0.020614909] ~= 0
    float x3[] = { 1, 1 };
    net.setInput(cv::Mat(1, 2, CV_32F, x3), "input:0");
    std::cout << "1 xor 1 = " << net.forward("output") << std::endl;

    return EXIT_SUCCESS;
}

Compile and Run:

$ sudo apt install build-essential pkg-config libopencv-dev
$ g++ predict.cpp `pkg-config --cflags --libs opencv4` -o predict
$ ./predict

Original Answer

The following simple example (XOR example) shows how to export Keras models (in both h5 format and pb format), and using the model in Python and C++:

train.py:

import numpy as np
import tensorflow as tf


def freeze_session(session, keep_var_names=None, output_names=None, clear_devices=True):
    """
    Freezes the state of a session into a pruned computation graph.

    Creates a new computation graph where variable nodes are replaced by
    constants taking their current value in the session. The new graph will be
    pruned so subgraphs that are not necessary to compute the requested
    outputs are removed.
    @param session The TensorFlow session to be frozen.
    @param keep_var_names A list of variable names that should not be frozen,
                          or None to freeze all the variables in the graph.
    @param output_names Names of the relevant graph outputs.
    @param clear_devices Remove the device directives from the graph for better portability.
    @return The frozen graph definition.
    """
    graph = session.graph
    with graph.as_default():
        freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
        output_names = output_names or []
        output_names += [v.op.name for v in tf.global_variables()]
        input_graph_def = graph.as_graph_def()
        if clear_devices:
            for node in input_graph_def.node:
                node.device = ''
        frozen_graph = tf.graph_util.convert_variables_to_constants(
            session, input_graph_def, output_names, freeze_var_names)
        return frozen_graph


X = np.array([[0,0], [0,1], [1,0], [1,1]], 'float32')
Y = np.array([[0], [1], [1], [0]], 'float32')

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(64, input_dim=2, activation='relu'))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(64, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

model.compile(loss='mean_squared_error', optimizer='adam', metrics=['binary_accuracy'])

model.fit(X, Y, batch_size=1, nb_epoch=100, verbose=0)

# inputs:  ['dense_input']
print('inputs: ', [input.op.name for input in model.inputs])

# outputs:  ['dense_4/Sigmoid']
print('outputs: ', [output.op.name for output in model.outputs])

model.save('./xor.h5')

frozen_graph = freeze_session(tf.keras.backend.get_session(), output_names=[out.op.name for out in model.outputs])
tf.train.write_graph(frozen_graph, './', 'xor.pbtxt', as_text=True)
tf.train.write_graph(frozen_graph, './', 'xor.pb', as_text=False)

predict.py:

import numpy as np
import tensorflow as tf

model = tf.keras.models.load_model('./xor.h5')

# 0 ^ 0 =  [[0.01974997]]
print('0 ^ 0 = ', model.predict(np.array([[0, 0]])))

# 0 ^ 1 =  [[0.99141496]]
print('0 ^ 1 = ', model.predict(np.array([[0, 1]])))

# 1 ^ 0 =  [[0.9897714]]
print('1 ^ 0 = ', model.predict(np.array([[1, 0]])))

# 1 ^ 1 =  [[0.00406971]]
print('1 ^ 1 = ', model.predict(np.array([[1, 1]])))

opencv-predict.py:

import numpy as np
import cv2 as cv


model = cv.dnn.readNetFromTensorflow('./xor.pb')

# 0 ^ 0 =  [[0.01974997]]
model.setInput(np.array([[0, 0]]), name='dense_input')
print('0 ^ 0 = ', model.forward(outputName='dense_4/Sigmoid'))

# 0 ^ 1 =  [[0.99141496]]
model.setInput(np.array([[0, 1]]), name='dense_input')
print('0 ^ 1 = ', model.forward(outputName='dense_4/Sigmoid'))

# 1 ^ 0 =  [[0.9897714]]
model.setInput(np.array([[1, 0]]), name='dense_input')
print('1 ^ 0 = ', model.forward(outputName='dense_4/Sigmoid'))

# 1 ^ 1 =  [[0.00406971]]
model.setInput(np.array([[1, 1]]), name='dense_input')
print('1 ^ 1 = ', model.forward(outputName='dense_4/Sigmoid'))

predict.cpp:

#include 
#include 
#include 

int main(int argc, char **argv)
{
    cv::dnn::Net net;

    net = cv::dnn::readNetFromTensorflow("./xor.pb");

    // 0 ^ 0 = [0.018541215]
    float x0[] = { 0, 0 };
    net.setInput(cv::Mat(1, 2, CV_32F, x0), "dense_input");
    std::cout << "0 ^ 0 = " << net.forward("dense_4/Sigmoid") << std::endl;

    // 0 ^ 1 = [0.98295897]
    float x1[] = { 0, 1 };
    net.setInput(cv::Mat(1, 2, CV_32F, x1), "dense_input");
    std::cout << "0 ^ 1 = " << net.forward("dense_4/Sigmoid") << std::endl;

    // 1 ^ 0 = [0.98810625]
    float x2[] = { 1, 0 };
    net.setInput(cv::Mat(1, 2, CV_32F, x2), "dense_input");
    std::cout << "1 ^ 0 = " << net.forward("dense_4/Sigmoid") << std::endl;

    // 1 ^ 1 = [0.010002014]
    float x3[] = { 1, 1 };
    net.setInput(cv::Mat(1, 2, CV_32F, x3), "dense_input");
    std::cout << "1 ^ 1 = " << net.forward("dense_4/Sigmoid") << std::endl;

    return EXIT_SUCCESS;
}

user_1734357 · Answer

At this time, all above older answers are outdated. As of Tensorflow 2.1

from tensorflow.keras.models import Model, load_model
model = load_model(MODEL_FULLPATH)
model.save(MODEL_FULLPATH_MINUS_EXTENSION)

will create a folder with a 'saved_model.pb' inside

user_4311535 · Answer

There is a very important point when you want to convert to tensorflow. If you use dropout, batch normalization or any other layers like these (which have not trainable but calculating values), you should change the learning phase of keras backend. Here is a discussion about it.

import keras.backend as K
k.set_learning_phase(0) # 0 testing, 1 training mode

user_1624666 · Answer

This solution worked for me. Courtesy to https://medium.com/tensorflow/training-and-serving-ml-models-with-tf-keras-fd975cc0fa27

import tensorflow as tf

# The export path contains the name and the version of the model
tf.keras.backend.set_learning_phase(0) # Ignore dropout at inference
model = tf.keras.models.load_model('./model.h5')
export_path = './PlanetModel/1'

# Fetch the Keras session and save the model
# The signature definition is defined by the input and output tensors
# And stored with the default serving key
with tf.keras.backend.get_session() as sess:
    tf.saved_model.simple_save(
        sess,
        export_path,
        inputs={'input_image': model.input},
        outputs={t.name:t for t in model.outputs})

user_5642366 · Answer

Please use tf.saved_model.simple_save, some example codes:

with tf.keras.backend.get_session() as sess:
    tf.saved_model.simple_save(
        sess,
        export_path,
        inputs={'input': keras_model.input},
        outputs={'output': keras_model.output})

===update====

You can use as_a_saved_model, example codes:

saved_model_path = tf.contrib.saved_model.save_keras_model(model, "./saved_models")

user_5465000 · Answer

If you want the model only for inference, you should first freeze the graph and then write it as a .pb file. The code snippet looks like this (code borrowed from here):

import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io
import keras
from keras import backend as K

sess = K.get_session()

constant_graph = graph_util.convert_variables_to_constants(
        sess,
        sess.graph.as_graph_def(),
        ["name_of_the_output_graph_node"])

graph_io.write_graph(constant_graph, "path/to/output/folder", 
                     "output_model_name", as_text=False)

You can do the above using the keras_to_tensorflow tool: https://github.com/amir-abdi/keras_to_tensorflow

The keras_to_tensorflow tool takes care of the above operations, with some extra features for a more diverse solution. Just call it with the correct input arguments (e.g. input_model and output_model flags).

If you want to retrain the model in tensorflow, use the above tool with the output_meta_ckpt flag to export checkpoints and meta graphs.

user_10997543 · Answer

using estimator.export_savedmodel we can easily convert h5 model to saved model. check doc here https://www.tensorflow.org/api_docs/python/tf/estimator/Estimator

def prepare_image(image_str_tensor):
    image_contents = tf.read_file(image_str_tensor)
    image = tf.image.decode_jpeg(image_contents, channels=3)
    image = tf.image.resize_images(image, [224, 224])
    image = tf.cast(image, tf.float32)
    return preprocess_input(image)

def serving_input_receiver_fn():
    input_ph = tf.placeholder(tf.string, shape=[None])
    images_tensor = tf.map_fn(
          prepare_image, input_ph, back_prop=False, dtype=tf.float32)
    images_tensor = tf.image.convert_image_dtype(images_tensor, 
                      dtype=tf.float32)

    return tf.estimator.export.ServingInputReceiver({"input": images_tensor}, 
             {'image_url': input_ph})

estimator = tf.keras.estimator.model_to_estimator(
    keras_model_path=h5_model_path
)

estimator.export_savedmodel(saved_model_path, serving_input_receiver_fn=serving_input_receiver_fn)

user_8079881 · Answer

Tensorflow tf.saved_model api is best for generating pb model

If you have h5 model then load it through keras load_model

from tensorflow import keras
model = keras.models.load_model("model.h5")

Save tensorflow model through saved_model api, It will save the model in pb format. This model will have required meta data for serving it through Google Ai Platform. So you can upload the directory to Ai Platform for serving your model.

import tensorflow as tf
tf.saved_model.save(model, './directory-to-save-file/')

user_7655027 · Answer

tf 2.2.0

import tensorflow.keras instead of just keras, because it will load your model as keras.engine.sequential.Sequential object which cannot be directly convertible into tensorflow .pb format

#import keras
import tensorflow.keras as keras
model = keras.models.load_model(load_path)
model.save(save_path)

user_5681083 · Answer

With tensorflow 2.x: If you want to save only the graph definition in pbtxt then use the below code.

import tensorflow as tf
keras_model = ...
tf.io.write_graph(
  keras_model.output.graph,
  'model_dir',
  'model.pbtxt',
  as_text=True,
)

user_9625777 · Answer

In the case of users trying to convert a Mask-RCNN model/weights into a frozen graph, most answers here won't suffice.

This can be done while saving the model (.h5) weights in the mrcnn/model.py file. Just need to make the following changes (git diff)

+    def freeze_session(self, session, keep_var_names=None, output_names=None, clear_devices=True):
+        """
+        Freezes the state of a session into a pruned computation graph.
+
+        Creates a new computation graph where variable nodes are replaced by
+        constants taking their current value in the session. The new graph will be
+        pruned so subgraphs that are not necessary to compute the requested
+        outputs are removed.
+        @param session The TensorFlow session to be frozen.
+        @param keep_var_names A list of variable names that should not be frozen,
+                              or None to freeze all the variables in the graph.
+        @param output_names Names of the relevant graph outputs.
+        @param clear_devices Remove the device directives from the graph for better portability.
+        @return The frozen graph definition.
+        """
+        graph = session.graph
+        with graph.as_default():
+            freeze_var_names = list(set(v.op.name for v in tf.global_variables()).difference(keep_var_names or []))
+            output_names = output_names or []
+            output_names += [v.op.name for v in tf.global_variables()]
+            input_graph_def = graph.as_graph_def()
+            if clear_devices:
+                for node in input_graph_def.node:
+                    node.device = ""
+            frozen_graph = tf.graph_util.convert_variables_to_constants(
+                session, input_graph_def, output_names, freeze_var_names)
+            return frozen_graph
+
     def train(self, train_dataset, val_dataset, learning_rate, epochs, layers,
               augmentation=None, custom_callbacks=None, no_augmentation_sources=None):
         """Train the model.
@@ -2373,6 +2401,12 @@ class MaskRCNN():
             workers=workers,
             use_multiprocessing=True,
         )
+        #######using session and saving .pb file##
+        frozen_graph = self.freeze_session(K.get_session(),
+                              output_names=[out.op.name for out in self.keras_model.outputs])
+        print('

		******* Writing Frozen Graph in logs directory *******

')
+        tf.train.write_graph(frozen_graph, self.log_dir, "my_model.pb", as_text=False)
+
         self.epoch = max(self.epoch, epochs)

The complete file can be found HERE. With it, I was able to convert ResNet50 and ResNet101 backbones for both coco as well as imagenet weights.

user_11745574 · Answer

In my case, I was trying to convert darknet weights to a TensorFlow model and I needed the model in .pb format. I tried so many solutions given here as well as on other forums, but I was finally able to fix it by upgrading my Tensorflow v2.2 to Tensorflow v2.3, and I could successfully save the model into a .pb format.

Here are the documentations for reference:

My imports:

import tensorflow as tf
import tensorflow.keras as keras

Code that saves the model in .pb format:

model.save("/path to directory/")

Code that saves the model in .h5 format:

tf.keras.models.save_model(model = model, filepath, modelname.h5')

Note: I could only get this working when I upgraded the Tensorflow from version 2.2 to 2.3

How to export Keras .h5 to tensorflow .pb?

14 Answers

Update for Tensorflow 2

Example for Tensorflow 2

Original Answer