EKS ingress-nginx and NLB with https redirect

user_6588456 0
Viewed 1498

I'm having issues with the nlb lately, it was quite an adventure to have nlb with https termination on the lb working with a redirection http=>https and an ingress-nginx on EKS.

Now, I want to have the X-Forwarded headers passed to the pod, but that breaks the http=>https redirection, I get a 400 on http requests.

On the service, I tried to put the service with http or tcp protocol, same thing.

Adding the service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*" header to the service, activates the proxy protocol v2 on all targets, and activating use-proxy-protocol: 'true' in the configmap for nginx breaks the http-snippet with the 308 redirection:

http-snippet: |
    server {
      listen 2443;
      return 308 https://$host$request_uri;
    }

Does anyone has a way to make it so that it can use the nlb with all the good header and the redirect working?

EDIT at comment request adding full working config

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/version: 0.41.0
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
data:
  http-snippet: |
    server {
      listen 2443 proxy_protocol;
      return 308 https://$host$request_uri;
    }
  proxy-real-ip-cidr: 10.4.0.0/16
  use-forwarded-headers: 'true'
  use-proxy-protocol: 'true'
  compute-full-forwarded-for: 'true'
aws-nlb ingress-nginx amazon-eks amazon-ec2
3 Answers

To conclude our comment discussion with @night-gold, to make NGINX to accept proxy protocol you have to specifically mention that in listen directive:

http {
    #...
    server {
        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;
        #...
    }
}

Check out NGINX guide for more.

user_113445020

Actually I tried to use that config with the HTTP snippet. But it didn't work. So searching about it a find the PR that solves the redirect with that config. The PR also exposes to the solution Port 2443 with the configuration tohttps in order to make the redirect work. I based on that one and work it for me.

https://github.com/kubernetes/ingress-nginx/pull/5374

https://github.com/kubernetes/ingress-nginx/pull/5374/files#diff-885b46a1b162f530aa95239e8c3adf9887a4ce863b443f49f06368011a4259ddR390-R393

user_128882510

In addition to this answer don't forget to add annotation of Proxy Protocol to service.

https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/service/nlb/#protocols

Here is my values in helm chart

helm upgrade \
      ingress-nginx ingress-nginx/ingress-nginx \
      --namespace ingress-nginx \
      --set controller.service.type=LoadBalancer \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"=nlb \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-nlb-target-type"=ip \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-cross-zone-load-balancing-enabled"=true \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"=external \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-name"="nginx-ingress" \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-proxy-protocol"="*" \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-scheme"="internet-facing" \
      --set controller.config."proxy-real-ip-cidr"="xx.xx.xx.xx/xx" \
      --set controller.config."use-forwarded-headers"="true" \
      --set controller.config."use-proxy-protocol"="true" \
      --set controller.config."compute-full-forwarded-for"="true" \
      --set controller.config."http-snippet"="
server{
  listen 2443;
  return 308 https://\$host\$request_uri;
}" \
      --dry-run
user_38250130
Related