I have trained a tf model (not using any sagemaker service). I used tf 2.1.0 version. I saved it with the following signature def MetaGraphDef with tag-set: 'serve' contains the following SignatureDef:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 80)
name: text_input_1:0
The given SavedModel SignatureDef contains the following output(s):
outputs['predictions'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 738)
name: output_1/Identity:0
Method name is: tensorflow/serving/predict
I zipped the model and created a Model object using AWS sagemaker's Python SDK
model = Model(model_data=s3_location,
role=role,
framework_version='2.1.0')
predictor = model.deploy(initial_instance_count=1, instance_type='ml.m5.xlarge') {code}
After the Model is created, I have a lot of trouble predicting. I get weird errors such as
"error": "Session was not created with a graph before Run()!" or
"error": "Failed to process element: 0 of 'instances' list. Error: Invalid argument: JSON Value:
I tried both predictor.predict() method as well as client.invoke_endpoint() methods using various formats such as predictor.predict(list), various variations of tf serving REST API formats using "instances" format. I tried json.dumps() on all variations as well. But nothing works.
Can someone suggest a way to predict. As shown in the signature_def my input is of shape (1,80).
Thank you
I'm looking for an answer of the form predictor.predict($what here?). Or suggest what I am doing wrong here.
Thank you