Do I need to add tf.compat.v1.disable_eager_execution() to export_inference_graph.py to convert tf.train.Checkpoint to SavedModel?

user_9400421 0
Viewed 258

I found a question about this error (in a different scenario), many Github issues and articles, but it seemingly always has to do with people upgrading from TF 1.x to TF 2.x. I'm not doing that.

Here are my versions:

  • tensorflow 2.5.0
  • tensorflow-addons 0.13.0
  • tensorflow-datasets 4.3.0
  • tensorflow-estimator 2.5.0
  • tensorflow-gpu 2.5.0

I'm trying to use TF object detection, converting a model trained in TF 2.5 via Python to a tensorflow.js compatible model and asked a question about it. The answer given was to start by running:

python export_inference_graph.py \
    --input_type image_tensor \
    --pipeline_config_path path/to/ssd_inception_v2.config \
    --trained_checkpoint_prefix path/to/model.ckpt \
    --output_directory path/to/exported_model_directory

So my command ended up being:

py Tensorflow\models\research\object_detection\export_inference_graph.py
--input_type image_tensor
--pipeline_config_path Tensorflow\workspace\models\my_ssd_mobnet\pipeline.config
--trained_checkpoint_prefix Tensorflow\workspace\pre-trained-models\ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8\checkpoint\ckpt-0.data-00000-of-00001
--output_directory Tensorflow\workspace\models\my_ssd_mobnet\export

Which resulted in the error:

RuntimeError: tf.placeholder() is not compatible with eager execution

I do see in the logs a common cause of this error, I know where it's coming from:

File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\object_detection-0.1-py3.8.egg\object_detection\exporter.py", line 186, in _image_tensor_input_placeholder
input_tensor = tf.placeholder(

But I don't understand how to deal with this, since I'm not writing any of these Tensorflow modules, I'm just trying to do something basic with existing modules, like converting a tf.train.Checkpoint to SavedModel.

Normally the answer seems to be to call tf.compat.v1.disable_eager_execution() but the weird thing about this is it's not my code, I don't know what else I'll potentially break in this conversion script by disabling a feature. Nor am I good enough with the Tensorflow API yet to really understand that script.

Full logs and trace:

2021-07-15 09:40:24.482953: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-07-15 09:40:26.835151: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll
2021-07-15 09:40:26.856379: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce RTX 2080 SUPER computeCapability: 7.5
coreClock: 1.845GHz coreCount: 48 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 462.00GiB/s
2021-07-15 09:40:26.856487: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll
2021-07-15 09:40:26.861810: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll
2021-07-15 09:40:26.861891: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll
2021-07-15 09:40:26.864685: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll
2021-07-15 09:40:26.865561: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll
2021-07-15 09:40:26.872246: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusolver64_11.dll
2021-07-15 09:40:26.874465: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll
2021-07-15 09:40:26.874979: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll
2021-07-15 09:40:26.875238: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-15 09:40:26.876220: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-07-15 09:40:26.877353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce RTX 2080 SUPER computeCapability: 7.5
coreClock: 1.845GHz coreCount: 48 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 462.00GiB/s
2021-07-15 09:40:26.877556: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-07-15 09:40:27.285985: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-15 09:40:27.286153: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264]      0
2021-07-15 09:40:27.286917: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0:   N
2021-07-15 09:40:27.287164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 5957 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 SUPER, pci bus id: 0000:01:00.0, compute capability: 7.5)
Traceback (most recent call last):
  File "Tensorflow\models\research\object_detection\export_inference_graph.py", line 206, in <module>
    tf.app.run()
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\absl\app.py", line 303, in run
    _run_main(main, args)
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\absl\app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "Tensorflow\models\research\object_detection\export_inference_graph.py", line 194, in main
    exporter.export_inference_graph(
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\object_detection-0.1-py3.8.egg\object_detection\exporter.py", line 611, in export_inference_graph
    _export_inference_graph(
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\object_detection-0.1-py3.8.egg\object_detection\exporter.py", line 503, in _export_inference_graph
    outputs, placeholder_tensor_dict = build_detection_graph(
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\object_detection-0.1-py3.8.egg\object_detection\exporter.py", line 457, in build_detection_graph
    placeholder_tensor, input_tensors = input_placeholder_fn_map[input_type](
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\object_detection-0.1-py3.8.egg\object_detection\exporter.py", line 186, in _image_tensor_input_placeholder
    input_tensor = tf.placeholder(
  File "C:\Users\jonat\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\ops\array_ops.py", line 3268, in placeholder
    raise RuntimeError("tf.placeholder() is not compatible with "
RuntimeError: tf.placeholder() is not compatible with eager execution.

What could I be doing here that would cause this error? Did I install the wrong version of the conversion script? I checked that I have the latest Tensorflow files from the official repo, and that's where export_inference_graph.py is found. Does the conversion script just not work with Tensorflow 2.x? Do I need to modify the conversion script with tf.compat.v1.disable_eager_execution()? Will this cause other problems in the script since I'm disabling a feature?

Edit:

I know some models in the object detection were built for tf 1.x (model zoo) and others 2.x (model zoo). I verified that I have a 2.x model, so that's not the cause.

tensorflow
1 Answers

Tensorflow allows you to save the model in multiple different format (checkpoint or savedmodel). The checkpoint just saves the weights for every layer so when loading the model, you need to first define the network architecture and then load the weights. The SavedModel saves the complete model i.e. architecture, weights and training configuration (including the optimizer weights). This link has more details related to the various format that are available.

https://www.tensorflow.org/tutorials/keras/save_and_load

In your case, since tfjs requires savedmodel as input, you can directly save the tensorflow model in the savedmodel rather than saving it first as checkpoint and then trying to convert it to savedmodel format.

user_128073620
Related