All GKE Cluster nodes restarted. How do we know the reason for the reboot?

Viewed 19

I've a GCP GKE cluster with version '1.22.12-gke.300'. There are around 20 nodes distributed in 4 node pools. The cluster was created a month before and last day I noticed that all of my nodes are rebooted/restarted. When I see the details of the node using command kubectl get nodes, I'm seeing this result (below). The age of all the nodes are 16 or 17 hours.

gke-company-name-gke-clust-company-name-default-n-97c8e50a-d63m   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-default-n-97c8e50a-l8zw   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-demo-app--d251216f-2uou   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-demo-app--d251216f-3mj1   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-demo-app--d251216f-doml   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-app--5ae07853-7mwd   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-app--5ae07853-gzxy   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-app--5ae07853-lgvo   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-27gf   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-7r4q   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-e680   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-m2vf   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-mtvg   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-mwiy   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-rwf9   Ready    <none>   16h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-tqe0   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-weai   Ready    <none>   17h     v1.22.12-gke.300
gke-company-name-gke-clust-company-name-prod-nifi-fd3e5533-xmss   Ready    <none>   16h     v1.22.12-gke.300

Is there any way to identify the reason behing this reboot/restart. Because this unexpected reboot caused some issues to my system. I just wanted to identify the reason of these reboots so that I can know that these types of reboots are expected in future.

Any help is appreciated.

1 Answers

Look for the log entries in /var/log/messages file. You would find some hints

Related