How to exclude some containers' metrics in Kubernetes Horizontal Pod Autoscaling

Question

I have a pod running with two containers. The actual application is running in one of the containers (container-app) and the other one is the proxy container (container-proxy). I enabled the Horizontal Pod Autoscaler (HPA) for CPU usage percentage but as it states in HPA documentation, both of the container metrics are put in the calculation.

I want to exclude the CPU metrics of container-proxy from HPA calculation because I want only application container to be the scaling element for the pod.

Is there any way to exclude some containers metrics from HPA calculation for multi-container pods?

user_11980539 · Answer

The cluster autoscaler works on a per-node pool basis. Horizontal Pod Autoscaler monitors CPU utilization of the pods and scales the number of replicas automatically. It provides immediate efficiency and capacity when needed, operates within user-defined minimum/maximum bounds, and allows users to set it and forget it. The design of the horizontal autoscaler is for pods not for the individual container.

HPA calculates pod cpu utilization as total cpu usage of all containers in pod divided by total request. It does not exclude containers metrics from HPA calculation if multiple containers are inside the pod.

user_288412 · Answer

Kubernetes 1.20+ supports container metrics, so as to target the utilisation per container, which would allow excluding a specific container of a pod from being considered.

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#container-resource-metrics

type: ContainerResource
containerResource:
  name: cpu
  container: application
  target:
    type: Utilization
    averageUtilization: 60

Its an alpha feature though, so not available without turning on alpha features in Kubernetes.

How to exclude some containers' metrics in Kubernetes Horizontal Pod Autoscaling

2 Answers