Is it possible to create an Estimator with arbitrarily many input Tensors for Predict SignatureDef without placeholders in TensorFlow 2.X?

Viewed 108

The Problem

I am converting my Tensorflow 1.14 estimator to TensorFlow 2.1. My current workflow involves training my tensorflow model on gcloud's ai-platform (training on gcloud) and using their model service to deploy my model for online predictions (model service).

The issue when upgrading to TensorFlow 2 is that they have done away with placeholders, which is affecting my serving_input_fn and how I export my estimator model. With tensorflow 2, if I export a model without the use of placeholders, my model's "predict" SignatureDef only has a single "examples" tensor whereas previously it had many inputs named appropriately through my serving_input_fn.

The previous set up for my estimator was as follows:

def serving_input_fn():

    inputs = {
        'feature1': tf.compat.v1.placeholder(shape=None, dtype=tf.string),
        'feature2': tf.compat.v1.placeholder(shape=None, dtype=tf.string),
        'feature3': tf.compat.v1.placeholder(shape=None, dtype=tf.string),
        ...
    }

    return tf.estimator.export.ServingInputReceiver(features=split_features, receiver_tensors=inputs)

exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)

eval_spec = tf.estimator.EvalSpec(
    input_fn=lambda: input_eval_fn(args.test_dir),
    exporters=[exporter],
    start_delay_secs=10,
    throttle_secs=0)

...

tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)

And this has worked fine in the past, it has allowed me to have a multi-input "predict" SignatureDef where I can send a json of the inputs to ai-platforms model service and get predictions back. But since I am trying to not rely on the tf.compat.v1 library, I want to avoid using placeholders.

What I've tried

Following the documentation linked here I've replaced my serving_input_fn with the tf.estimator.export.build_parsing_serving_input_receiver_fn method:

feature_columns = ... # list of feature columns 
serving_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(
  tf.feature_column.make_parse_example_spec(feature_columns))

However, this gives me the following "predict" SignatureDef:

signature_def['predict']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['examples'] tensor_info:
        dtype: DT_STRING
        shape: (-1)
        name: input_example_tensor:0

whereas before my "predict" SignatureDef was as follows:

signature_def['predict']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['feature1'] tensor_info:
        dtype: DT_STRING
        shape: unknown_rank
        name: Placeholder:0
    inputs['feature2'] tensor_info:
        dtype: DT_STRING
        shape: unknown_rank
        name: Placeholder_1:0
    inputs['feature3'] tensor_info:
        dtype: DT_STRING
        shape: unknown_rank
        name: Placeholder_2:0

I've also tried using the tf.estimator.export.build_raw_serving_input_receiver_fn, but my understanding is that this method requires actual Tensors in order to be used instead of a feature spec. Unless I use placeholders, I don't really understand where to grab these serving Tensors from.

So my main questions are:

  • Is it possible to create a multi-input "predict" signature def from an estimator model without using placeholders in Tensorflow 2?
  • If it is not possible, how am I supposed to provide the instances to gcloud predictions service for the "examples" tensor in the "predict" signature def?

Thanks!

0 Answers
Related