I'm newbie to Tensorflow/Keras so any kind of help is highly appreciated.
I've trained an U-Net Deep Learning Neural Network in Matlab and, then, exported it to ONNX. To create a Keras model, I've used the onnx_to_keras function. The idea is to use the model just for inference and I've verified that I get the same results both in Matlab and Keras. As I've got stuck trying to create the Protocol Buffer (.pb) file for Tensorflow, I've also previously saved the model to a .hdf5 file. Afterwards, I've loaded back the model and also check that it correctly predicts (I've to use the load_model function from tensorflow.keras.models because, otherwise, I get an error).
In order to convert the Keras model to a .pb file, I've followed the steps of a Xilinx Vitis AI tutorial. When a I created a model from scratch in Keras/Tensorflow those steps did not produce any errors so it has something to do with the imported model itself, I think.
The error message is:
Traceback (most recent call last):
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
return fn(*args)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
target_list, run_metadata)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Error while reading resource variable Encoder_Section_2_Conv_2/bias from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/Encoder_Section_2_Conv_2/bias)
[[{{node Encoder_Section_2_Conv_2/bias/Read/ReadVariableOp}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Keras2TF_Jon_2.py", line 84, in <module>
save_path = saver.save(sess, os.path.join(cfg.CHKPT_MODEL_DIR, "unet/float_model.ckpt")) #Aqu\xed almacenamos los detalles sin el esqueleto
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 1193, in save
raise exc
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 1176, in save
{self.saver_def.filename_tensor_name: checkpoint_file})
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 956, in run
run_metadata_ptr)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
run_metadata)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Error while reading resource variable Encoder_Section_2_Conv_2/bias from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/Encoder_Section_2_Conv_2/bias)
[[node Encoder_Section_2_Conv_2/bias/Read/ReadVariableOp (defined at /opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]
Original stack trace for 'Encoder_Section_2_Conv_2/bias/Read/ReadVariableOp':
File "Keras2TF_Jon_2.py", line 50, in <module>
model = onnx_to_keras(onnx_model, ['imageInputLayer'], change_ordering=True)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/onnx2keras/converter.py", line 181, in onnx_to_keras
keras_names
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/onnx2keras/convolution_layers.py", line 177, in convert_conv
layers[node_name] = conv(input_0)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 824, in __call__
self._maybe_build(inputs)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2146, in _maybe_build
self.build(input_shapes)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/layers/convolutional.py", line 174, in build
dtype=self.dtype)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 529, in add_weight
aggregation=aggregation)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/training/tracking/base.py", line 712, in _add_variable_with_custom_getter
**kwargs_for_getter)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer_utils.py", line 139, in make_variable
shape=variable_shape if variable_shape else None)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py", line 258, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call
shape=shape)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py", line 197, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/variable_scope.py", line 2503, in default_variable_creator
shape=shape)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py", line 262, in __call__
return super(VariableMetaclass, cls).__call__(*args, **kwargs)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py", line 1406, in __init__
distribute_strategy=distribute_strategy)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py", line 1587, in _init_from_args
value = gen_resource_variable_ops.read_variable_op(handle, dtype)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_resource_variable_ops.py", line 587, in read_variable_op
"ReadVariableOp", resource=resource, dtype=dtype, name=name)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
op_def=op_def)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
attrs, op_def, compute_device)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
op_def=op_def)
File "/opt/vitis_ai/conda/envs/vitis-ai-tensorflow/lib/python3.6/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in __init__
self._traceback = tf_stack.extract_stack()
IMPORTANT: The layer associated to the error changes from execution to execution: Encoder_Section_2_Conv_1_1/kernel, Decoder_Section_1_Conv_1_1/kernel, Encoder_Section_1_Conv_1_1/kernel...
The code that produces the error is provided in the next lines:
config = tf.compat.v1.ConfigProto()
set_session(tf.compat.v1.Session(config=config))
assert os.path.isdir(cfg.ONNX_MODEL_DIR)
# Load ONNX model
onnx_model = onnx.load('/workspace/tutorials/VAI-KERAS-FCN8-SEMSEG/files/ONNX_model/unet.onnx')
# Call the converter (input - is the main model input name, can be different for your model)
m_model = onnx_to_keras(onnx_model, ['imageInputLayer'], change_ordering=True)
# set learning phase for no training: This line must be executed before loading Keras model
K.set_learning_phase(0)
saver = tf.compat.v1.train.Saver()
sess = K.get_session()
#Saving the model means saving all the values of the parameters and the graph
save_path = saver.save(sess, os.path.join(cfg.CHKPT_MODEL_DIR, "unet/float_model.ckpt"))
By the way, just in case, the structure of the model is the following one:
Layer (type) Output Shape Param # Connected to
imageInputLayer (InputLayer) [(None, 128, 128, 25 0
Encoder_Section_1_Conv_1_pad (Z (None, 130, 130, 25) 0 imageInputLayer[0][0]
Encoder_Section_1_Conv_1 (Conv2 (None, 128, 128, 8) 1808 Encoder_Section_1_Conv_1_pad[0][0
Encoder_Section_1_LeakyReLU_1 ( (None, 128, 128, 8) 0 Encoder_Section_1_Conv_1[0][0]
Encoder_Section_1_Conv_2_pad (Z (None, 130, 130, 8) 0 Encoder_Section_1_LeakyReLU_1[0][
Encoder_Section_1_Conv_2 (Conv2 (None, 128, 128, 8) 584 Encoder_Section_1_Conv_2_pad[0][0
Encoder_Section_1_LeakyReLU_2 ( (None, 128, 128, 8) 0 Encoder_Section_1_Conv_2[0][0]
Encoder_Section_1_MaxPool (MaxP (None, 64, 64, 8) 0 Encoder_Section_1_LeakyReLU_2[0][
Encoder_Section_2_Conv_1_pad (Z (None, 66, 66, 8) 0 Encoder_Section_1_MaxPool[0][0] --> Mode 1a error: "Encoder_Section_2_Conv_1_1/kernel"
Encoder_Section_2_Conv_1 (Conv2 (None, 64, 64, 16) 1168 Encoder_Section_2_Conv_1_pad[0][0
Encoder_Section_2_LeakyReLU_1 ( (None, 64, 64, 16) 0 Encoder_Section_2_Conv_1[0][0]
Encoder_Section_2_Conv_2_pad (Z (None, 66, 66, 16) 0 Encoder_Section_2_LeakyReLU_1[0][
Encoder_Section_2_Conv_2 (Conv2 (None, 64, 64, 16) 2320 Encoder_Section_2_Conv_2_pad[0][0
Encoder_Section_2_LeakyReLU_2 ( (None, 64, 64, 16) 0 Encoder_Section_2_Conv_2[0][0]
Encoder_Section_2_DropOut (Drop (None, 64, 64, 16) 0 Encoder_Section_2_LeakyReLU_2[0][
Encoder_Section_2_MaxPool (MaxP (None, 32, 32, 16) 0 Encoder_Section_2_DropOut[0][0]
Mid_Conv_1_pad (ZeroPadding2D) (None, 34, 34, 16) 0 Encoder_Section_2_MaxPool[0][0]
Mid_Conv_1 (Conv2D) (None, 32, 32, 32) 4640 Mid_Conv_1_pad[0][0]
Mid_LeakyReLU_1 (LeakyReLU) (None, 32, 32, 32) 0 Mid_Conv_1[0][0]
Mid_Conv_2_pad (ZeroPadding2D) (None, 34, 34, 32) 0 Mid_LeakyReLU_1[0][0]
Mid_Conv_2 (Conv2D) (None, 32, 32, 32) 9248 Mid_Conv_2_pad[0][0]
Mid_LeakyReLU_2 (LeakyReLU) (None, 32, 32, 32) 0 Mid_Conv_2[0][0]
Mid_DropOut (Dropout) (None, 32, 32, 32) 0 Mid_LeakyReLU_2[0][0]
Decoder_Section_1_UpConv (Conv2 (None, 64, 64, 16) 2064 Mid_DropOut[0][0]
Decoder_Section_1_LeakyUpReLU ( (None, 64, 64, 16) 0 Decoder_Section_1_UpConv[0][0]
Decoder_Section_1_DepthConcaten (None, 64, 64, 32) 0 Decoder_Section_1_LeakyUpReLU[0][ Encoder_Section_2_DropOut[0][0]
Decoder_Section_1_Conv_1_pad (Z (None, 66, 66, 32) 0 Decoder_Section_1_DepthConcatenat
Decoder_Section_1_Conv_1 (Conv2 (None, 64, 64, 16) 4624 Decoder_Section_1_Conv_1_pad[0][0
Decoder_Section_1_LeakyReLU_1 ( (None, 64, 64, 16) 0 Decoder_Section_1_Conv_1[0][0]
Decoder_Section_1_Conv_2_pad (Z (None, 66, 66, 16) 0 Decoder_Section_1_LeakyReLU_1[0][
Decoder_Section_1_Conv_2 (Conv2 (None, 64, 64, 16) 2320 Decoder_Section_1_Conv_2_pad[0][0
Decoder_Section_1_LeakyReLU_2 ( (None, 64, 64, 16) 0 Decoder_Section_1_Conv_2[0][0]
Decoder_Section_2_UpConv (Conv2 (None, 128, 128, 8) 520 Decoder_Section_1_LeakyReLU_2[0][
Decoder_Section_2_LeakyUpReLU ( (None, 128, 128, 8) 0 Decoder_Section_2_UpConv[0][0]
Decoder_Section_2_DepthConcaten (None, 128, 128, 16) 0 Decoder_Section_2_LeakyUpReLU[0][ Encoder_Section_1_LeakyReLU_2[0][
Decoder_Section_2_Conv_1_pad (Z (None, 130, 130, 16) 0 Decoder_Section_2_DepthConcatenat
Decoder_Section_2_Conv_1 (Conv2 (None, 128, 128, 8) 1160 Decoder_Section_2_Conv_1_pad[0][0
Decoder_Section_2_LeakyReLU_1 ( (None, 128, 128, 8) 0 Decoder_Section_2_Conv_1[0][0]
Decoder_Section_2_Conv_2_pad (Z (None, 130, 130, 8) 0 Decoder_Section_2_LeakyReLU_1[0][
Decoder_Section_2_Conv_2 (Conv2 (None, 128, 128, 8) 584 Decoder_Section_2_Conv_2_pad[0][0
Decoder_Section_2_LeakyReLU_2 ( (None, 128, 128, 8) 0 Decoder_Section_2_Conv_2[0][0]
Final_ConvolutionLayer (Conv2D) (None, 128, 128, 3) 27 Decoder_Section_2_LeakyReLU_2[0][
Thank you in advance to everyone!