How can I scale-in Sagemaker endpoint?

Question

I have deployed a sklearn model in AWS Sagemaker using sklearn.deploy method, for auto-scaling the endpoint. I've set the following configuration:

Target value for number of requests: 25
Scale out cool time: 30 sec
Scale in cool time: 20 sec

After receiving sending 25+ requests a new instance is deployed. But after this even when I don't send new requests to the endpoint it is not scaling down automatically.

Why is it not scaling down?

How can I make it auto-scale down when no new requests are received for a fixed time interval.

user_2596715 · Answer

As of the writing of this post, SageMaker will not scale down to 0.

You must also specify the minimum number of instances for the model. This value must be at least 1, and equal to or less than the value specified for the maximum number of endpoint instances.

Source: https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-auto-scaling-prerequisites.html

How can I scale-in Sagemaker endpoint?

1 Answers