We have Openshift cluster 3.11, where we have configured Horizontal Autoscaler:
Min Pods: 5
Max Pods:10
Cpu Request Target: 80%
1) We have a spring application running in pods which persist data to mongodb. I have observed that when processing starts, it does increase pod count. However it also starts terminating pods when CPU utilization is less.
I have observed that when pods are terminated, it does not consider any processing running on that pod. Due to the same, i do find few database records are lost(records which were about to be saved from the pod which got terminated): mongodb collection count.
2) When i ran with fixed number of pods, i do not loose data.
Is there any way to tune Autoscaler so that it does not terminate pod , if there is any processing running ( for e.g async database save operations) on the pod
Or Terminate the pod only after particular duration even though the CPU utilization less?