Receiving parse error from SageMaker Multi Model Endpoint using TensorFlow

Viewed 112

We are currently moving our models from single model endpoints to multi model endpoints within AWS SageMaker. After deploying the Multi Model Endpoint using prebuilt TensorFlow containers I receive the following error when calling the predict() method:

{"error": "JSON Parse error: The document root must not be followed by other value at offset: 17"}

I invoke the endpoint like this:

data = np.random.rand(n_samples, n_features)

predictor = Predictor(endpoint_name=endpoint_name)
prediction = predictor.predict(data=serializer.serialize(data), target_model=model_name)

My function for processing the input is the following:

def _process_input(data, context):
    data = data.read().decode('utf-8')
    data = [float(x) for x in data.split(',')]
    return json.dumps({'instances': [data]})

For the training I configured my container as follows:

tensorflow_container = TensorFlow(
    entry_point=path_script,
    framework_version='2.4',
    py_version='py37',
    instance_type='ml.m4.2xlarge',
    instance_count=1,
    role=EXECUTION_ROLE,
    sagemaker_session=sagemaker_session,
    hyperparameters=hyperparameters)

tensorflow_container.fit()

For deploying the endpoint I first initializing a Model from a given Estimator and then a MultiDataModel:

model = estimator.create_model(
            role=EXECUTION_ROLE, 
            image_uri=estimator.training_image_uri(),
            entry_point=path_serving)

mdm = MultiDataModel(
        name=endpoint_name, 
        model_data_prefix=dir_model_data,
        model=model, 
        sagemaker_session=sagemaker.Session())

mdm.deploy(
        initial_instance_count=1, 
        instance_type=instance_type,
        endpoint_name=endpoint_name)


Afterwards the single models are added using:

mdm.add_model(
    model_data_source=source_path,
    model_data_path=model_name)

Thank you for any hints and help.

1 Answers

This issue usually occurs in case you either have damaged or malformed JSON data. Recommend you running it past JSON validator https://jsonlint.com/

I work at AWS and my opinions are my own - Thanks,Raghu

Related