ingress-nginx Errors connection refused

Question

I am trying to access my ingress-nginx service from a service but it gives connection refused. Here is my ingress

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
    - host: ticketing.dev
      http:
        paths:
          - path: /api/users/?(.*)
            backend:
              serviceName: auth-srv
              servicePort: 3000
          - path: /api/tickets/?(.*)
            backend:
              serviceName: tickets-srv
              servicePort: 3000
          - path: /?(.*)
            backend:
              serviceName: client-srv
              servicePort: 3000

apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: http
      port: 443
      protocol: TCP
      targetPort: https

❯ kubectl get services -n ingress-nginx
NAME            TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)                      AGE
ingress-nginx   LoadBalancer   10.101.124.218   10.101.124.218   80:30634/TCP,443:30179/TCP   15m

The ingress-nginx is running on namespace ingress-nginx. So it should be accessible by http://ingress-nginx.ingress-nginx.svc.cluster.local. But when I access it, it says connection refused 10.101.124.218:80. I am able to access the ingress from outside, i.e. from the ingress ip.

I am using minikube and used ingress by running minikube addons enable ingress. Yes and im running the tunnel by minikube tunnel

user_12524159 · Answer

I tested your environment and found the same behavior, external access but internally getting connection refused, this is how I solved:

The Minikube Ingress Addon deploys the controller in kube-system namespace. If you try to deploy the service in a newly created namespace, it will not reach the deployment in kube-system namespace.
It's easy to mix those concepts because the default nginx-ingress deployment uses the namespace ingress-nginx as you were trying.
Another issue I found, is that your service does not have all selectors assigned to the controller deployment.
The easiest way to make your deployment work, is to run kubectl expose on the nginx controller:

kubectl expose deployment ingress-nginx-controller --target-port=80 --type=NodePort -n kube-system

Using this command to create the nginx-ingress-controller service, all communications were working, both external and internal.

Reproduction:

For this example I'm using only two ingress backends to avoid being much repetitive in my explanation.
Using minikube 1.11.0
Enabled ingress and metallb addons.
Deployed two hello apps: v1 and v2, both pods listens on port 8080 and are exposed as node port as follows:

$ kubectl get services
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
hello1-svc   NodePort    10.110.211.119           8080:31243/TCP   95m
hello2-svc   NodePort    10.96.9.66               8080:31316/TCP   93m

Here is the ingress file, just like yours, just changed the backend services names and ports to match my deployed ones:

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  rules:
    - host: ticketing.dev
      http:
        paths:
          - path: /api/users/?(.*)
            backend:
              serviceName: hello1-svc
              servicePort: 8080
          - path: /?(.*)
            backend:
              serviceName: hello2-svc
              servicePort: 8080

Now I'll create the nginx-ingress service exposing the controller deployment, this way all tags and settings will be inherited:

$ kubectl expose deployment ingress-nginx-controller --target-port=80 --type=NodeP
ort -n kube-system
service/ingress-nginx-controller exposed

Now we deploy the ingress object:

$ kubectl apply -f ingress.yaml 
ingress.networking.k8s.io/ingress-service created

$ kubectl get ingress
NAME              CLASS    HOSTS           ADDRESS      PORTS   AGE
ingress-service      ticketing.dev   172.17.0.4   80      56s

$ minikube ip
172.17.0.4

Testing the ingress from the outside:

$ tail -n 1 /etc/hosts
172.17.0.4 ticketing.dev

$ curl http://ticketing.dev/?foo
Hello, world!
Version: 2.0.0
Hostname: hello2-67bbbf98bb-s78c4

$ curl http://ticketing.dev/api/users/?foo
Hello, world!
Version: 1.0.0
Hostname: hello-576585fb5f-67ph5

Then I deployed a alpine pod to test the access from inside the cluster:

$ kubectl run --generator=run-pod/v1 -it alpine --image=alpine -- /bin/sh
/ # nslookup ingress-nginx-controller.kube-system.svc.cluster.local
Server:         10.96.0.10
Address:        10.96.0.10:53

Name:   ingress-nginx-controller.kube-system.svc.cluster.local
Address: 10.98.167.112

/ # apk update
/ # apk add curl

/ # curl -H "Host: ticketing.dev" ingress-nginx-controller.kube-system.svc.cluster.local/?foo
Hello, world!
Version: 2.0.0
Hostname: hello2-67bbbf98bb-s78c4

/ # curl -H "Host: ticketing.dev" ingress-nginx-controller.kube-system.svc.cluster.local/api/users/?foo
Hello, world!
Version: 1.0.0
Hostname: hello-576585fb5f-67ph5

As you can see, all requests were fulfilled.

Note:

As pointed by @suren, when curling ingress, I had to specify the host with -H
The service name needs to be fully FQDN because we are dealing with a service hosted in another namespace, using the format ..svc.cluster.local.
In your JS app, you will have to pass the Host argument in order to reach the ingress.

If you have any question let me know in the comments.

ingress-nginx Errors connection refused

1 Answers