Kubernetes does not respect initialDelaySeconds when starting up

user_2409793 0
Viewed 297

I have configured a pod as follows:

  livenessProbe:
    initialDelaySeconds: 60
    periodSeconds: 10
    failureThreshold: 3
  readinessProbe:
    initialDelaySeconds: 60
    periodSeconds: 10
    failureThreshold: 3

Readiness and Liveness probes are tcp-socker based

          readinessProbe:
            tcpSocket:
              port: {{ .Values.port }}
            initialDelaySeconds: {{ .Values.readinessProbe.initialDelaySeconds }}
            periodSeconds: {{ .Values.readinessProbe.periodSeconds }}
            failureThreshold: {{ .Values.readinessProbe.failureThreshold }}

However, when I deploy, the pod is marked almost immediately as failed and goes in Error --> CrashLoopBackOff. The pod checks a redis connection (which does indeed need a little time to become ready).

And of course I see the connection errors in my pod's logs

  File "/usr/local/lib/python3.9/site-packages/redis/connection.py", line 563, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to redis-master:6379. Connection refused.

Why is the pod being marked in Error / CLBO so eagerly, way before initialDelaySeconds: 60?

Here is the pod's yaml dump regarding the probes (I increased the initialDelaySecond to both probes to 100, but still the same...)

    livenessProbe:
      failureThreshold: 3
      initialDelaySeconds: 100
      periodSeconds: 10
      successThreshold: 1
      tcpSocket:
        port: 9898
      timeoutSeconds: 1
    name: mycontainer
    ports:
    - containerPort: 9898
      protocol: TCP
    readinessProbe:
      failureThreshold: 3
      initialDelaySeconds: 100
      periodSeconds: 10
      successThreshold: 1
      tcpSocket:
        port: 9898
      timeoutSeconds: 1
kubernetes
1 Answers

initialDelaySeconds: Number of seconds after the container has started before liveness or readiness probes are initiated.

Now pod can get failed because of CrashLoopBackOff even before starting the probes. This is the concept here. It can occur if you set the container restartPolicy to Never.

You can see the pod logs or events for getting the reason of pod failure (can use kubectl describe <pod>)

user_73604170
Related