Multiple ingress controllers using same host

user_12296095 0
Viewed 7090

I have two ingress controllers (for public/internal traffic), what I would like is for all endpoints to use the public ingress except for /metrics, which should be internal, all using the same host.

E.g.

example.com/ -> public ingress
example.com/metrics -> internal ingress

This is what I have tried:

internal ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-metrics-ingress
  annotations:
    kubernetes.io/ingress.class: ingress-internal
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /metrics
        backend:
          serviceName: example-servicename
          servicePort: 80

and public ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path:
        backend:
          serviceName: example-servicename
          servicePort: 80

The internal ingress is currently being ignored when I visit example.com/metrics (it uses the public one instead).

If I change the internal ingress to use the same ingress controller as the public one and change the service port to 81 (as an example), this provides an error (which is expected), this shows that the two different ingresses are being used. However, as soon as I use two different ingress controllers, then the one ingress' rules are not being picked up.

How can I configure my ingresses to achieve my desired result?

nginx-ingress kubernetes-ingress kubernetes
2 Answers

When running multiple ingress-nginx controllers, it will only process an unset class annotation if one of the controllers uses the default --ingress-class value (see IsValid method in internal/ingress/annotations/class/main.go), otherwise the class annotation become required.

If --ingress-class is set to the default value of nginx, the controller will monitor Ingresses with no class annotation and Ingresses with annotation class set to nginx. Use a non-default value for --ingress-class, to ensure that the controller only satisfied the specific class of Ingresses.

In your case use the combination of the annotation kubernetes.io/ingress.class: "EXTERNAL|INTERNAL" and the flag --ingress-class=EXTERNAL|INTERNAL allows you to filter which Ingress rules should be picked by the nginx ingress controller.

Take a look: multiple-ingress, ingress-nginx-traffic.

user_113003820

I have had an issue like that on AKS (K8s version 1.22.4). I have two Nginx Ingress Controllers, Internal and External.

Only one worked at a time, Internal or external.

After specifying a unique election-id for each one the problem was fixed.

The following post may help: https://stackoverflow.com/a/72591382/4049017

user_40490170
Related