How can I wait for a Google App Engine operation to finish?

Viewed 1861

In my CI/CD process I deploy a GAE application and then delete old versions.

Sometimes there's a conflict where a deploy is in the process of deleting a version and then a second deploy comes along and tries to delete the same version, causing an error like this:

Cannot operate on apps/my-gae-service/services/backend/versions/20200713t161545 because an operation is already in progress for apps/my-gae-service/services/core-api/versions/20200713t161545 by 642f0fbd-9633-4aeb-b1cf-71b52c9fdf45.

What is the best way to wait for all GAE operations to complete before running a command?

This could be done with dependencies between build steps but that doesn't always work because of an issue that appears to be on the GAE side. The issue is that sometimes versions deleted from one CLI command aren't totally gone even after that command is done running. I'm opening a ticket about that with Google soo.

1 Answers

You can check if there is an operation in progress with this command:

gcloud app operations list --filter=status=PENDING --format=json

You filter only on the PENDING operation. If there is no operation in progress, the return is [], else, you have the details (in format JSON, you have more detail than in the tabular view).

Here you can see that I update a service "method": "google.appengine.v1.Services.UpdateService",

[
  {
    "id": "56f9cb4f-4381-4314-9557-f3c676e64f69",
    "op_resource": {
      "metadata": {
        "@type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1",
        "insertTime": "2020-07-14T10:13:17.839Z",
        "method": "google.appengine.v1.Services.UpdateService",
        "target": "apps/ProjectID/services/default",
        "user": "myUser@test.com"
      },
      "name": "apps/ProjectID/operations/56f9cb4f-4381-4314-9557-f3c676e64f69"
    },
    "project": "ProjectID",
    "start_time": "2020-07-14T10:13:17.839Z",
    "status": "PENDING"
  }
]

Now you can perform a check in your CI to avoid conflict, and you can know on which operation is the conflict.

Related