Assigning version labels in model_config_file fails

user_10806241 0
Viewed 917

I have versions 1 and 2 of a model and I'm trying to assign them labels, following the instructions at https://www.tensorflow.org/serving/serving_config#assigning_string_labels_to_model_versions_to_simplify_canary_and_rollback

I have exported the two versions in /path/to/model/1 and /path/to/model/2 respectively and I'm starting the server with the following command:

tensorflow_model_server --rest_api_port=8501 --model_config_file=models.config

The following models.config file works, and results in only serving version 1 (and if the specific message is omitted, version 2 is served as expected since it corresponds to the highest number):

model_config_list {
    config {
        name: 'm1'
        base_path: '/path/to/model/'
        model_platform: 'tensorflow'
        model_version_policy {
        specific {
            versions: 1
        }
    }
}

I have verified that I can use the server to send requests to the model and perform inference as expected. However if I try to add version_labels by using this config file:

model_config_list {
    config {
        name: 'm1'
        base_path: '/path/to/model/'
        model_platform: 'tensorflow'
        model_version_policy {
        specific {
            versions: 1
        }
        version_labels {
            key: 'current'
            value: 1
        }
    }
}

then launching the server fails with the following error:

Failed to start server. Error: Failed precondition: Request to assign label to version 1 of model m1, which is not currently available for inference.

I've also noticed that changing the value field to a non-existent version folder yields a similar result:

Failed to start server. Error: Failed precondition: Request to assign label to version 1234 of model m1, which is not currently available for inference.

I'm using:

TensorFlow ModelServer: 1.12.0-rc0+dev.sha.87470f0
TensorFlow Library: 1.12.0

I couldn't find any SO questions on the topic of version_labels and the available tensorflow documentation seems incomplete and outdated (for instance it doesn't mention the need to pass model_platform: 'tensorflow' in the config file).

Any help would be much appreciated!

tensorflow-serving tensorflow
2 Answers

refer to https://www.tensorflow.org/tfx/serving/serving_config

Please note that labels can only be assigned to model versions that are loaded and available for serving. Once a model version is available, one may reload the model config on the fly, to assign a label to it (can be achieved using HandleReloadConfigRequest RPC endpoint).

Maybe you should delete the label related part first, then start the tensorflow serving, and finally add the label related part to the config file on the fly.

user_79685980

More recently, it is possible to assign labels to models that are not yet loaded provided the --allow_version_labels_for_unavailable_models=true flag is specified (in the same place you specify your --model_config_file=... variable. (In my case, I do this in a docker-compose.yaml file.)

For more information, see the official documentation, which at the time of writinng says,

If you would like to assign a label to a version that is not yet loaded (for ex. by supplying both the model version and the label at startup time) then you must set the --allow_version_labels_for_unavailable_models flag to true, which allows new labels to be assigned to model versions that are not loaded yet.

user_40932780
Related