What are the practical differences between an IP vs instance based target types for an AWS NLB?

user_260574 0
Viewed 3067

I'm using an AWS NLB to expose my Kubernetes pods to the internet. The NLB is currently using instance target types but I noticed there are also IP target types. What are the differences between an instance target type vs an IP target type from a practical point of view? When would you use one over the other?

AWS's documentation specifies some restrictions around using IP target types, namely they must be within certain CIDRs and constraints around maximums, but I'm trying to understand when you might want to use one over the other.

I don't know if it has any impact, but we've been having issues with our kubernetes rollingUpdate deployments where we're seeing downtime for the pods as they switch over (we have liveness and readiness checks there).

kubernetes amazon-elb amazon-web-services
2 Answers

The three key use-cases for using IP target type:

  • your target does not have to be an instance - anything with private IP address will work, including internal load balance, VPC private service, Fargate containers, databases, on-premise servers through VPN.
  • your target can be in different region, as long as you have cross-region peering between your VPCs
  • you have multiple network interfaces on your instance, so you can load distribute traffic between them, e.g. different applications on a single instance are bind to different interfaces. Each interface can be associated with different target group.

Instance target type is only limited instances. It should be your default choice when load balancing instances. For example, if you have instances in autoscaling group (ASG), the ASG can automatically register your instances with your load balancer. You can't do this for IP target types.

user_2488230

In the context of Amazon Conainer Network Interface (CNI), Each Elastic Network Interface (ENI) on the node has a range of how many secondary IP addresses it can support. Pods are allocated IP addresses from this secondary Ip address range (that's the job of CNI).

When this range of secondary IP addresses is exhausted (maybe too many pods or too volatile pods), the CNI provision a new secondary ENI on the Node with a new range of secondary IP addresses for the pod (Depending on type of EC2 instace it has a limit to the number of ENIs that it can support).

As per NLB request routing.

If you specify targets using an instance ID, traffic is routed to instances using the primary private IP address that is specified in the primary network interface for the instance.

So now there is an impact on Pods density. And maybe that's happening with the rolling updates, that it exceeds the maximum limit.

user_52529600
Related