How to create only internal load balancer with ingress-nginx chart?

user_3484180 0
Viewed 5686

In my AWS EKS, I have installed nginx-ingress with following command:

helm upgrade --install -f controller.yaml \
    --namespace nginx-ingress \
    --create-namespace \
    --version 3.26.0 \
    nginx-ingress ingress-nginx/ingress-nginx

Where controller.yaml file looks like this:

controller:
  ingressClass: nginx-internal
  service:
    internal:
      enabled: true
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0

I have few applications, and individual ingresses per application with different virtual hosts and I want all ingress objects point to internal load balancer, Even if I set ingressClass in ingresses of applications, It seems they point to Public Load balancer:

kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx-internal

So, is there a way to create only single internal load balancer with its ingresses pointing to that load balancer ?

Thanks

amazon-eks kubernetes-helm nginx-ingress
4 Answers

I managed to get this working by using the following controller.yaml

controller:
  ingressClassByName: true
  
  ingressClassResource:
    name: nginx-internal
    enabled: true
    default: false
    controllerValue: "k8s.io/ingress-nginx-internal"

  service:
    # Disable the external LB
    external:
      enabled: false

    # Enable the internal LB. The annotations are important here, without
    # these you will get a "classic" loadbalancer
    internal:
      enabled: true
      annotations:
        service.beta.kubernetes.io/aws-load-balancer-internal: "true"
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
        service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
        service.beta.kubernetes.io/aws-load-balancer-type: nlb

Then you can use the ingressClassName as follows

kind: Ingress
spec:
  ingressClassName: nginx-internal

It's not necessary but I deployed this to a namespace that reflected the internal only ingress

helm upgrade --install \
  --create-namespace ingress-nginx-internal ingress-nginx/ingress-nginx \
  --namespace ingress-nginx-internal \
  -f controller.yaml
user_14006640

Noticed in your controller.yaml that you enabled internal setup. According to documentation, this setup creates two load balancers, an external and an internal, in case you want to expose some applications to internet and others only inside your vpc in same k8s cluster.

If you want just one internal load balancer, try to setup you controller.yaml like this:

controller:
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-internal: "true" 
      service.beta.kubernetes.io/aws-load-balancer-subnets: "subnet-xxxxx,subnet-yyyyy,subnet-zzzzz"

It will provision just one NBL that routes the traffic internally.

Using service.beta.kubernetes.io/aws-load-balancer-subnets annotation, you can choose which Availability Zones / Subnets your load balancer will routes traffic to.

If you remove service.beta.kubernetes.io/aws-load-balancer-type annotation, a Classic Load Balancer will be provisioned instead of Network.

user_111221240

Based on @rmakoto answer, it seems some configs are missing, in order to tell AWS to create an internal NLB. I've tried with the following configs, and now it works like expected:

controller:
  service:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-name: "k8s-nlb"
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
      service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
      service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
      service.beta.kubernetes.io/aws-load-balancer-subnets: "subnet-xxx,subnet-yyy,subnet-zzz"

Now to deploy run the following command:

helm upgrade --install \
  --create-namespace ingress-nginx nginx-stable/nginx-ingress \
  --namespace ingress-nginx \
  -f controller.yaml
user_26960490

If you only want classic ELBs. This worked for me.

  controller:
    service:
      external:
        enabled: false
      internal:
        enabled: true
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-internal: "true"
user_32636500
Related