How to expose port 22 via NGINX Ingress Controller?

user_721666 0
Viewed 4360

I have a Kubernetes Cluster with a NGINX Ingress Controller. In the cluster I have deployed a Gitea POD. The Web UI and the HTTPS access is exposed via an Ingress object like this one:

---
kind: Service
apiVersion: v1
metadata:
  name: gitea-service
  namespace: gitea-repo
spec:
  selector:
    app: gitea
  ports:
  - name: gitea-http
    port: 3000
  - name: gitea-ssh
    port: 22
---
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: git-tls
  namespace: gitea-repo
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - git.foo.com
    secretName: tls-gitea
  rules:
  - host: git.foo.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: gitea-service
            port:
              number: 3000

This works all fine for HTTPS.

But Gitea also provides a SSH access through port 22. My Question is, how can I tell the NGINX Ingress Controller to route also port 22 to my pod?

As far as I understand I should patch my NGINX controller deployment with something like that:

spec:
  template:
    spec:
      containers:
      - name: controller
        # defind cusotm tcp/udp configmap
        args:
        - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-configmap-giteassh

        ports:
         - name: ssh
           containerPort: 22
           protocol: TCP

and providing a config map pointing to my gitea service:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-configmap-giteassh
  namespace: ingress-nginx
data:
  22: "gitea-repo/gitea-service:22"

Do I need also an additional Ingress configuration in my Gitea POD now?

I wonder if this is the correct approach. Why I am forced to define this in the NGINX controller and not in my POD namespace as I do for HTTP? This would mean that for each POD exposing a TCP port other than HTTP I have to adjust my Ingress NGINX Controller?

kubernetes nginx
1 Answers

I decided to improve my answer by adding more details and explanations.
I found your blog post which contains all the information I need to reproduce this problem.

In your example you need TCP traffic to pass through on port 22 (it is TCP protocol).
NGINX Ingress Controller doesn't support TCP protocol, so additional configuration is necessary, which can be found in the documentation.
You can follow the steps below to expose the TCP service:

  1. Create a ConfigMap with the specified TCP service configuration.
  2. Add the --tcp-services-configmap flag to the Ingress controller configuration.
  3. Expose port 22 in the Service defined for the Ingress.

Ad 1. We need to create a ConfigMap with the key that is the external port to use and the value that indicates the service to expose (in the format <namespace/service name>:<service port>:[PROXY]:[PROXY]).
NOTE: You may need to change Namespace depending on where the NGINX Ingress controller is deployed.

$ cat ingress-nginx-tcp.yml
apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-nginx-tcp
  namespace: default  
data:
  "22": gitea-repo/gitea-service:22

Ad 2. After creating the ConfigMap, we can point to it using the --tcp-services-configmap flag in the Ingress controller configuration.
NOTE: Additionally, if you want to use a name (e.g. 22-tcp) in the port definition and then reference this name in the targetPort attribute of a Service, you need to define port 22 (see: Defining a Service documentation).

$ kubectl get deployment ingress-nginx-controller -oyaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ingress-nginx-controller
  namespace: default
spec:
...
  template:
...
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --tcp-services-configmap=$(POD_NAMESPACE)/ingress-nginx-tcp
...

Ad 3. Then we need to expose port 22 in the Service defined for the Ingress.

$ kubectl get svc -oyaml ingress-nginx-controller
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx-controller
  namespace: default
spec:
  ports:
  - name: 22-tcp
    nodePort: 30957
    port: 22
    protocol: TCP
    targetPort: 22
...
  type: LoadBalancer
...

Finally, we can check if it works as expected by creating a new repository on the command line:
Note: We need to have a gitea user with the proper SSH keys associated with this account.

$ git add README.md
$ git commit -m "first commit"
[master (root-commit) c6fa042] first commit
 1 file changed, 0 insertions(+), 0 deletions(-)
 create mode 100644 README.md
$ git remote add origin git@<PUBLIC_IP>:<USERNAME>/repository.git
$ git push -u origin master
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 211 bytes | 211.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: . Processing 1 references
remote: Processed 1 references in total
To <PUBLIC_IP>:<USERNAME>/repository.git
 * [new branch]      master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.

In addition, we can log to the NGINX Ingress Controller Pod and check if it is listening on port 22:

$ kubectl exec -it ingress-nginx-controller-784d4c9d9-jhvnm -- bash
bash-5.1$ netstat -tulpn | grep 22
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 :::22                   :::*                    LISTEN      -
user_148012250
Related