Kubernetes Pod anti-affinity - evenly spread pods based on a label?

user_857994 0
Viewed 1393

We are finding that our Kubernetes cluster tends to have hot-spots where certain nodes get far more instances of our apps than other nodes.

In this case, we are deploying lots of instances of Apache Airflow, and some nodes have 3x more web or scheduler components than others.

Is it possible to use anti-affinity rules to force a more even spread of pods across the cluster?

E.g. "prefer the node with the least pods of label component=airflow-web?"

If anti-affinity does not work, are there other mechanisms we should be looking into as well?

kubernetes affinity scheduling
3 Answers

Try adding this to the Deployment/StatefulSet .spec.template:

      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: "component"
                  operator: In
                  values:
                  - airflow-web
              topologyKey: "kubernetes.io/hostname"
user_345860

Have you tried configuring the kube-scheduler?

kube-scheduler selects a node for the pod in a 2-step operation:

  • Filtering: finds the set of Nodes where it's feasible to schedule the Pod.
  • Scoring: ranks the remaining nodes to choose the most suitable Pod placement.

Scheduling Policies: can be used to specify the predicates and priorities that the kube-scheduler runs to filter and score nodes.

kube-scheduler --policy-config-file <filename>

One of the priorities for your scenario is:

  • BalancedResourceAllocation: Favors nodes with balanced resource usage.
user_110320440
user_39556010
Related