The Problem
I am converting my Tensorflow 1.14 estimator to TensorFlow 2.1. My current workflow involves training my tensorflow model on gcloud's ai-platform (training on gcloud) and using their model service to deploy my model for online predictions (model service).
The issue when upgrading to TensorFlow 2 is that they have done away with placeholders, which is affecting my serving_input_fn and how I export my estimator model. With tensorflow 2, if I export a model without the use of placeholders, my model's "predict" SignatureDef only has a single "examples" tensor whereas previously it had many inputs named appropriately through my serving_input_fn.
The previous set up for my estimator was as follows:
def serving_input_fn():
inputs = {
'feature1': tf.compat.v1.placeholder(shape=None, dtype=tf.string),
'feature2': tf.compat.v1.placeholder(shape=None, dtype=tf.string),
'feature3': tf.compat.v1.placeholder(shape=None, dtype=tf.string),
...
}
return tf.estimator.export.ServingInputReceiver(features=split_features, receiver_tensors=inputs)
exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)
eval_spec = tf.estimator.EvalSpec(
input_fn=lambda: input_eval_fn(args.test_dir),
exporters=[exporter],
start_delay_secs=10,
throttle_secs=0)
...
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
And this has worked fine in the past, it has allowed me to have a multi-input "predict" SignatureDef where I can send a json of the inputs to ai-platforms model service and get predictions back. But since I am trying to not rely on the tf.compat.v1 library, I want to avoid using placeholders.
What I've tried
Following the documentation linked here I've replaced my serving_input_fn with the tf.estimator.export.build_parsing_serving_input_receiver_fn method:
feature_columns = ... # list of feature columns
serving_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
tf.feature_column.make_parse_example_spec(feature_columns))
However, this gives me the following "predict" SignatureDef:
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['examples'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: input_example_tensor:0
whereas before my "predict" SignatureDef was as follows:
signature_def['predict']:
The given SavedModel SignatureDef contains the following input(s):
inputs['feature1'] tensor_info:
dtype: DT_STRING
shape: unknown_rank
name: Placeholder:0
inputs['feature2'] tensor_info:
dtype: DT_STRING
shape: unknown_rank
name: Placeholder_1:0
inputs['feature3'] tensor_info:
dtype: DT_STRING
shape: unknown_rank
name: Placeholder_2:0
I've also tried using the tf.estimator.export.build_raw_serving_input_receiver_fn, but my understanding is that this method requires actual Tensors in order to be used instead of a feature spec. Unless I use placeholders, I don't really understand where to grab these serving Tensors from.
So my main questions are:
- Is it possible to create a multi-input "predict" signature def from an estimator model without using placeholders in Tensorflow 2?
- If it is not possible, how am I supposed to provide the instances to gcloud predictions service for the "examples" tensor in the "predict" signature def?
Thanks!