How to detect if a Kubernetes application is truly "ready"?

Viewed 287

I have a web app on Kubernetes that gets "spun-up" via user action. That user action triggers a python script that applies a deployment, service, and ingress. The frontend shows a spinner until the app is ready to make connections. I'm currently checking the status of the deployment and checking that the "Available" status is "True", at which point I hide the spinner and load the app.

The problem is, every once in a while, users will experience a 503: Temporarily Unavailable error. A quick browser refresh fixes the problem, so this appears to be some sort of race condition.

My question is, why am I getting a 503 error if the deployment is marked as "Available"? Does this mean that the ingress or the service is sometimes taking longer to initialize?

I currently have the following probes on my deployment's app container:

      readinessProbe:
        httpGet:
          path: /
          port: 3000
          periodSeconds: 5
          initialDelaySeconds: 5
      livenessProbe:
        httpGet:
          path: /
          port: 3000
          periodSeconds: 5
          initialDelaySeconds: 5

I'm using Azure AKS and ingress-nginx.

1 Answers

For newly created deployments check these. Both should be true, order does not matter.

  • kubectl get deployment <name> -ojson | jq ".status.availableReplicas" equal to desired, or >= 1, on your preference.
  • kubectl get ingress <name> -ojson | jq ".status.loadBalancer" is not empty. It means that ingress controller initialized for your host.

For updated deployments (anything that required pods to be recreated). Both should be true, order does not matter.

  • kubectl get deployment <name> -ojson | jq ".status.availableReplicas" equal to desired.

  • kubectl get deployment <name> -ojson | jq ".status.updatedReplicas" equal to desired.

    Ingress will already be initialized here.

Related