Kubernetes EKS deployment set soft node affinity to split pods 50/50 per nodegroup

user_11496618 0
Viewed 450

I have an EKS cluster with two nodegroups each in different AZ. One deployment Deployment1 is running on 2 namespaces for redundancy, one copy per namespace and each of them run in separate AZs/nodegroup. Also there is another deployment Deployment2 that does not have any node affinity set and K8s manages where pods get scheduled. Both deployments are huge with lots of pods. I have a subnet of 250 IPs available to me for each node group.

The problem is that while Deployment1 is fine on it's own, and gets split almost equally per AZ/Nodegroup, the Deployment2 tends to schedule most pods in one of the nodegroups and that ends when there are no more IPs available. This is a problem for Deployment1 since one namespace of it is tied to that nodegroup and no new pods can be scheduled there if load changes.

Can I somehow balance Deployment2 so it has 'soft affinity' that would split it 50/50 per each nodegroup, but if needed, can schedule pods in the other nodegroup?

kubernetes-deployment amazon-eks kubernetes-pod kubernetes
1 Answers

If you're using Kubernetes 1.19 or later you can use topologySpreadConstraints, adding this to the pod template:

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        foo: bar

where maxSkew define how uneven pods can be scheduled, topologyKey is the key of node labels and the labelSelector matches a label of your deployment. See docs

If your on an older Kubernetes version, you can look at pod anti affinity.

user_100204190
Related