CloudRun is going over max instance count (of 1)

user_8430249 0
Viewed 752

I have a cloud run service (receiving no traffic) where I set the max instances and min instances to 1 so that it's always running.

When deploying a new instance, the instance count jumps to 3. This is a problem (I make some requests on instance start that hits a 429 if two instances are simultaneously making these requests).

Why is CloudRun instance count going over my max?

I can confirm my settings are correct and looking at the logs there are two new instances that start up.

PS: Cloudrun does have this message, which makes me think what I'm trying to do isn't possible. I just figured it would be because of downtime instead of extra instances.

Revisions using a maximum number of instances of 3 or less might experience unexpected downtime.

google-cloud-run google-cloud-platform
2 Answers

I read that it is a cloud run problem you need 10 instances and then the error disappears.

I understood that it is a bug in golang that does not know how to do it with two instances, but with 10 instances it should be fixed.

https://github.com/ahmetb/cloud-run-faq/issues/54

user_147714230

Your scenario seems to fit one described in the documentation, in which a new deployment for a Cloud Run service might temporarily create additional instances:

When you deploy a new revision, Cloud Run gradually migrates traffic from the old revision to the new one. Because maximum instance limits are set for each revision, you may temporarily exceed the specified limit during the period after deployment.

Additional instances might be created to handle traffic migration from the previous revision to a new one. Cloud Run offers settings that can alter how migration occurs between revisions. One of these settings is used to instantly serve all new traffic on the new revision. You can test if using this setting helps reduce the number of instances that are created. I tested one of the provided sample services and created multiple revisions, which did not exceed 1 active instance.

user_169298940
Related