Why am I getting inconsistent results when attempting to update my instance group using `gcloud`?

Viewed 352

I have an instance group in GCP, and I am working on automating the deployment process. The instances in this group are based on a tagged GCR image. When a new image is pushed to the container registry, we have been manually triggering an upgrade by navigating to the instance group from console.cloud.google.com, clicking "restart/replace vms", and setting these options:

  • Operation: replace
  • Maximum surge: 3
  • Maximum unavailable: 0

Here is my gcloud command for doing the same thing (link to Google's documentation about this command):

gcloud beta compute instance-groups managed rolling-action start-update my-instance-group \
    --version=template=my-template-with-image \
    --replacement-method=substitute \
    --max-surge=3 \
    --max-unavailable=0 \
    --region=us-central1

Manually, the process always works. But the gcloud command is flaky. It always appears to succeed from the command line, but the instance groups are not always restarted. I have even tried adding these two flags, and the restart attempt was still unreliable:

    --minimal-action=replace \
    --most-disruptive-allowed-action=replace \

There is quite a lot of output from the gcloud command (which I can provide, if necessary), but here are the only parts of the output that differ between a successful and unsuccessful attempt:

Good:

currentActions:
  creating: 1
status:
  isStable: false
  versionTarget:
    isReached: false

Bad:

currentActions:
  creating: 0
status:
  isStable: true
  versionTarget:
    isReached: true

That is pretty much the extent of my knowledge at this point. I am not sure how to move forward in automating the build process, and I have been unable to find answers from the documentation so far.

I hope I was not too verbose, and thank you in advance to anyone who spends time on this :)

0 Answers
Related