Use iptables to block all Kubernetes nodeport communication from outside of cluster

user_624724 0
Viewed 876

For security reasons, I would like to block all of my kubeadm based on-site Kubernetes worker nodes from being able to forward nodePort traffic.

I tried applying on a worker the obvious:

iptables -I INPUT 1 -p tcp -m multiport  --dports 30000:32767 -j DROP

but it doesn't appear to work.

I also tried the same thing on -I FORWARD 1 but the service appears open still.

any suggestions?

# iptables -L INPUT --line-numbers -n
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination
1    ACCEPT     all  --  127.0.0.1            0.0.0.0/0
2    REJECT     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:10250 reject-with icmp-port-unreachable
3    KUBE-SERVICES  all  --  0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes service portals */
4    KUBE-EXTERNAL-SERVICES  all  --  0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes externally-visible service portals */
5    KUBE-FIREWALL  all  --  0.0.0.0/0            0.0.0.0/0
6    WEAVE-NPC-EGRESS  all  --  0.0.0.0/0            0.0.0.0/0

# iptables -L FORWARD --line-numbers -n
Chain FORWARD (policy DROP)
num  target     prot opt source               destination
1    WEAVE-NPC-EGRESS  all  --  0.0.0.0/0            0.0.0.0/0            /* NOTE: this must go before '-j KUBE-FORWARD' */
2    WEAVE-NPC  all  --  0.0.0.0/0            0.0.0.0/0            /* NOTE: this must go before '-j KUBE-FORWARD' */
3    NFLOG      all  --  0.0.0.0/0            0.0.0.0/0            state NEW nflog-group 86
4    DROP       all  --  0.0.0.0/0            0.0.0.0/0
5    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
6    ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
7    KUBE-FORWARD  all  --  0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
8    KUBE-SERVICES  all  --  0.0.0.0/0            0.0.0.0/0            ctstate NEW /* kubernetes service portals */
9    DOCKER-USER  all  --  0.0.0.0/0            0.0.0.0/0
10   DOCKER-ISOLATION-STAGE-1  all  --  0.0.0.0/0            0.0.0.0/0
11   ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
12   DOCKER     all  --  0.0.0.0/0            0.0.0.0/0
13   ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
14   ACCEPT     all  --  0.0.0.0/0            0.0.0.0/0
kubernetes iptables
1 Answers

It's best practices to run only Kubernetes on a dedicated server. Running any side applications or tuning applications that are used by the cluster may cause an impact in the Kubernetes itself and isn't recommended/supported.

Depending on your setup, kubernetes uses iptables, so any changes you make can impact kubernetes and also be ruined by kubernetes.

You have a few options to achieve what you need, one is to have an external firewall protecting your server, if you can't have I suggest you to use tools that where created for that purpose.

The best option in my opinion is to use Calico as a your CNI as it has features that allow you to block specific traffic. Please refer to Apply Calico policy to Kubernetes node ports.

Example: (not tested)

  apiVersion: projectcalico.org/v3
  kind: GlobalNetworkPolicy
  metadata:
    name: deny-nodeports
  spec:
    applyOnForward: true
    preDNAT: true
    ingress:
    - action: Deny
      destination:
        ports:
        - 30000:32767
      protocol: TCP
      source: {}
    - action: Deny
      destination:
        ports:
        - 30000:32767
      protocol: UDP
      source: {}
    selector: has(kubernetes-host)
    order: 1100
    types:
    - Ingress

Another option is to use Network Policies, a network policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints.

And finally one last option I would recommend is Istio. Istio is for sure the most complex solution but in the other hand, it's the most powerful.

user_121535760
Related