Knative: Why one Dockerfile "Readiness probe failed:" and another do not?

Viewed 372

kubectl version --short

Client Version: v1.22.2
Server Version: v1.21.1

This Dockerfile give "Readiness probe failed":

# syntax=docker/dockerfile:1

FROM golang:1.17.2-alpine as builder

WORKDIR /source

COPY ./src/go.mod ./
# COPY ./src/go.sum ./
RUN go mod download

COPY ./src ./
# COPY src/*.go ./

# output app in app folder
RUN go build -o /app

FROM golang:1.16-alpine
COPY --from=builder /app /app
EXPOSE 80
# execute app
CMD [ "/app" ]

Deploying a simple knative service:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
  namespace: default
spec:
  template:
    spec:
      containers:
        - image: gcr.io/<projid>/dockerfile-micro-depend:latest

Error

kubectl describe pod dockerfile-micro-depend-dhv8n-deployment-796484d788-hwpmm

Name:         dockerfile-micro-depend-dhv8n-deployment-796484d788-hwpmm
Namespace:    default
Priority:     0
Node:         kind-control-plane/172.18.0.2
Start Time:   Fri, 10 Dec 2021 17:37:26 +0100
Labels:       app=dockerfile-micro-depend-dhv8n
              pod-template-hash=796484d788
              serving.knative.dev/configuration=dockerfile-micro-depend
              serving.knative.dev/configurationGeneration=1
              serving.knative.dev/revision=dockerfile-micro-depend-dhv8n
              serving.knative.dev/revisionUID=053a72d2-c506-43cd-9c3c-bbd3e7296295
              serving.knative.dev/service=dockerfile-micro-depend
Annotations:  serving.knative.dev/creator: kubernetes-admin
Status:       Running
IP:           10.244.0.12
IPs:
  IP:           10.244.0.12
Controlled By:  ReplicaSet/dockerfile-micro-depend-dhv8n-deployment-796484d788
Containers:
  user-container:
    Container ID:   containerd://c90f6129263426a5ec58fc2f4286a6b5ae85faa82f86f50e05c5126ca8082e4c
    Image:          gcr.io/<projid>/dockerfile-micro-depend@sha256:dc9b331e5e63af3067578652c8822af91c73c19226541c3495080d0049328452
    Image ID:       gcr.io/<projid>/dockerfile-micro-depend@sha256:dc9b331e5e63af3067578652c8822af91c73c19226541c3495080d0049328452
    Port:           8080/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 10 Dec 2021 17:37:32 +0100
    Ready:          True
    Restart Count:  0
    Environment:
      PORT:                      8080
      K_REVISION:                dockerfile-micro-depend-dhv8n
      K_CONFIGURATION:           dockerfile-micro-depend
      K_SERVICE:                 dockerfile-micro-depend
      K_INTERNAL_POD_NAME:       dockerfile-micro-depend-dhv8n-deployment-796484d788-hwpmm (v1:metadata.name)
      K_INTERNAL_POD_NAMESPACE:  default (v1:metadata.namespace)
    Mounts:
      /var/log from knative-var-log (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-plmnm (ro)
  queue-proxy:
    Container ID:   containerd://4448ee3b78b9a18ef7f2132eb444c943fd6cc471dd36f04cee6ce3aa4b685bc5
    Image:          gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:713bd548700bf7fe5452969611d1cc987051bd607d67a4e7623e140f06c209b2
    Image ID:       gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:713bd548700bf7fe5452969611d1cc987051bd607d67a4e7623e140f06c209b2
    Ports:          8022/TCP, 9090/TCP, 9091/TCP, 8012/TCP
    Host Ports:     0/TCP, 0/TCP, 0/TCP, 0/TCP
    State:          Running
      Started:      Fri, 10 Dec 2021 17:37:32 +0100
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:      25m
    Readiness:  exec [/ko-app/queue -probe-period 0] delay=0s timeout=10s period=1s #success=1 #failure=3
    Environment:
      SERVING_NAMESPACE:                      default
      SERVING_SERVICE:                        dockerfile-micro-depend
      SERVING_CONFIGURATION:                  dockerfile-micro-depend
      SERVING_REVISION:                       dockerfile-micro-depend-dhv8n
      QUEUE_SERVING_PORT:                     8012
      CONTAINER_CONCURRENCY:                  0
      REVISION_TIMEOUT_SECONDS:               300
      SERVING_POD:                            dockerfile-micro-depend-dhv8n-deployment-796484d788-hwpmm (v1:metadata.name)
      SERVING_POD_IP:                          (v1:status.podIP)
      SERVING_LOGGING_CONFIG:                 {
                                                "level": "info",
                                                "development": false,
                                                "outputPaths": ["stdout"],
                                                "errorOutputPaths": ["stderr"],
                                                "encoding": "json",
                                                "encoderConfig": {
                                                  "timeKey": "ts",
                                                  "levelKey": "level",
                                                  "nameKey": "logger",
                                                  "callerKey": "caller",
                                                  "messageKey": "msg",
                                                  "stacktraceKey": "stacktrace",
                                                  "lineEnding": "",
                                                  "levelEncoder": "",
                                                  "timeEncoder": "iso8601",
                                                  "durationEncoder": "",
                                                  "callerEncoder": ""
                                                }
                                              }
      SERVING_LOGGING_LEVEL:                  
      SERVING_REQUEST_LOG_TEMPLATE:           
      SERVING_REQUEST_METRICS_BACKEND:        prometheus
      TRACING_CONFIG_BACKEND:                 none
      TRACING_CONFIG_ZIPKIN_ENDPOINT:         
      TRACING_CONFIG_STACKDRIVER_PROJECT_ID:  
      TRACING_CONFIG_DEBUG:                   false
      TRACING_CONFIG_SAMPLE_RATE:             0.1
      USER_PORT:                              8080
      SYSTEM_NAMESPACE:                       knative-serving
      METRICS_DOMAIN:                         knative.dev/internal/serving
      DOWNWARD_API_LABELS_PATH:               /etc/podinfo/labels
      SERVING_READINESS_PROBE:                {"tcpSocket":{"port":8080,"host":"127.0.0.1"},"successThreshold":1}
      ENABLE_PROFILING:                       false
      SERVING_ENABLE_PROBE_REQUEST_LOG:       false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-plmnm (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  knative-var-log:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  kube-api-access-plmnm:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age               From               Message
  ----     ------     ----              ----               -------
  Normal   Scheduled  82s               default-scheduler  Successfully assigned default/dockerfile-micro-depend-dhv8n-deployment-796484d788-hwpmm to kind-control-plane
  Normal   Pulling    81s               kubelet            Pulling image "gcr.io/<projid>/dockerfile-micro-depend@sha256:dc9b331e5e63af3067578652c8822af91c73c19226541c3495080d0049328452"
  Normal   Pulled     77s               kubelet            Successfully pulled image "gcr.io/<projid>/dockerfile-micro-depend@sha256:dc9b331e5e63af3067578652c8822af91c73c19226541c3495080d0049328452" in 4.6859265s
  Normal   Created    76s               kubelet            Created container user-container
  Normal   Started    76s               kubelet            Started container user-container
  Normal   Pulled     76s               kubelet            Container image "gcr.io/knative-releases/knative.dev/serving/cmd/queue@sha256:713bd548700bf7fe5452969611d1cc987051bd607d67a4e7623e140f06c209b2" already present on machine
  Normal   Created    76s               kubelet            Created container queue-proxy
  Normal   Started    76s               kubelet            Started container queue-proxy
  Warning  Unhealthy  5s (x7 over 66s)  kubelet            Readiness probe failed:

Working:

Then I found this "official" example and used the Dockerfile, and it worked. Do you know why?

Clearly it has to do with some health checking on the pod, but I have no idea why the first Dockerfile is not working.

1 Answers

It looks like your application runs on port 80 (based on the EXPOSE 80 line in your Dockerfile), but the default for Knative is to run on 8080 (exposed as $PORT to your container).

You can override this by setting a (single) containerPort in your yaml to point Knative at the port your application serves on.

Related