We have created an endpoint in Vertex AI. We have got the client library route working. However, we also want to figure out the gRPC route since that is closest to the gRPC route we had with self managed TF-Serving. Can someone provide a code pointer for Vertex AI model serving using gRPC (preferably in Python)?
1 Answers
gRPC can be used through Vertex Prediction private endpoint, but it is not yet officially supported. See sample here: https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/vertex_endpoints/optimized_tensorflow_runtime/tabular_optimized_online_prediction.ipynb
Related