How to detect GKE autoupgrading a node in Stackdriver logs

Viewed 3228

We have a GKE cluster with auto-upgrading nodes. We recently noticed a node become unschedulable and eventually deleted that we suspect was being upgraded automatically for us. Is there a way to confirm (or otherwise) in Stackdriver that this was indeed the cause what was happening?

2 Answers

You can use the following advanced logs queries with Cloud Logging (previously Stackdriver) to detect upgrades to node pools:

protoPayload.methodName="google.container.internal.ClusterManagerInternal.UpdateClusterInternal"
resource.type="gke_nodepool"

and master:

protoPayload.methodName="google.container.internal.ClusterManagerInternal.UpdateClusterInternal"
resource.type="gke_cluster"

Additionally, you can control when the update are applied with Maintenance Windows (like the user aurelius mentioned).

I think your question has been already answered in the comments. Just as addition automatic upgrades occur at regular intervals at the discretion of the GKE team. To get more control you can create a Maintenance Windows as explained here. This is basically a time frame that you choose in which automatic upgrades should occur.

Related