Graceful shutdown of tomcat on a kubernetes cluster

user_7898224 0
Viewed 3485

I am running a java application on tomcat in a k8s cluster, I am using a prestop lifecycle hook to run /usr/local/tomcat/bin/shutdown.sh before the container gets terminated. Also defining a termination grace period of 60 secs.

expected behaviour: tomcat shutsdown gracefully when I kill a pod or delete a deployment.

actual behaviour: tomcat shutsdown abruptly.

Any thoughts? Thanks in advance. 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample
  annotations:
spec:
  replicas: 2
  selector:
    matchLabels:
      app: sample
      name: sample
  template:
    metadata:
      labels:
        app: sample
        name: sample
    spec:
      hostname: sample-web
      terminationGracePeriodSeconds: 60
      volumes:
       - name: splunk-inputs
         configMap:
           name: splunk-conf
           items:
           - key: inputs.conf
             path: ./inputs.conf
       - name: splunk-outputs
         configMap:
           name: splunk-conf
           items:
           - key: outputs.conf
             path: ./outputs.conf
       - name: docker-socket
         hostPath:
          path: /var/run/docker.sock
       - name: tomcat-log-common
         emptyDir: {} 
      containers:
      - name: sample
        image: sampleregistery.azurecr.io/root
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command: ["/usr/local/tomcat/bin/shutdown.sh"]
kubernetes docker tomcat java
1 Answers

found the issue:

I was running this prestop lifecycle hook:

lifecycle: preStop: exec: command: ["/usr/local/tomcat/bin/catalina.sh"]

It should be something like:

lifecycle: preStop: exec: command: ["/usr/local/tomcat/bin/catalina.sh && sleep 30"]

In the first case, K8s was attempting to exit out of the lifecycle hook as soon as it invokes the shutdown.sh without waiting for tomcat to drain all threads.

it just takes about 5-10 secs depending upon your app. giving a sleep of 30 secs will keep the prestop hook active which should give enough time for tomcat to finish the shutdown process" .

user_78982240
Related