AWS EKS, How To Hit Pod directly from browser?

user_5053886 0
Viewed 1854

I'm very new to kubernetes. I have spent the last week learning about Nodes, Pods, Clusters, Services, and Deployments.

With that I'm trying to just get some more understanding of how the networking for kubernetes even works. I just want to expose a simple nginx docker webpage and hit it from my browser.

Our VPC is setup with a direct connect so I'm able to hit EC2 instances on their private IP addresses. I also setup the EKS cluster using the UI on aws for now as private. For testing purposes I have added my cidr range to be allowed on all TCP as an additional security group in the EKS cluster UI.

Here is my basic service and deployment definitions:

apiVersion: v1
kind: Service
metadata:
  name: testing-nodeport
  namespace: default
  labels:
    infrastructure: fargate
    app: testing-app
spec:
  type: NodePort
  selector:
    app: testing-app
  ports:
    - port: 80
      targetPort: testing-port
      protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: testing-deployment
  namespace: default
  labels:
    infrastructure: fargate
    app: testing-app
spec:
  replicas: 1
  selector:
    matchLabels:
      infrastructure: fargate
      app: testing-app
  template:
    metadata:
      labels:
        infrastructure: fargate
        app: testing-app
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - name: testing-port
          containerPort: 80

I can see that everything is running correctly when I run:

kubectl get all -n default

However, when I try to hit the NodePort IP address on port 80 I can't load it from the browser.

I can hit the pod if I first setup a kubectl proxy at the following url (as the proxy is started on port 8001):

http://localhost:8001/api/v1/namespaces/default/services/testing-nodeport:80/proxy/

I'm pretty much lost at this point. I don't know what I'm doing wrong and why I can't hit the basic nginx docker outside of the kubectl proxy command.

kubernetes docker amazon-web-services
3 Answers

What if you use the proxy option? Something like this:

kubectl port-forward -n default service/testing-nodeport 3000:80
Forwarding from 127.0.0.1:3000 -> 80
Forwarding from [::1]:3000 -> 80

After this, you can access your K8S service from localhost:3000. More info here

user_45652340

Imagine that the kubernetes cluster is like your AWS VPC. It has its own internal network with private IPs and connects all the PODs. Kubernetes only exposes things which you explicitly ask to expose.

Service port 80 is available within the cluster. So one pod can talk to this service using the service name:service port. But if you need to access from outside, you need ingress controller / LoadBalancer. You can also use NodePort for testing purposes. The node port will be something bigger than 30000 (within this 30000-32767).

You should be able to access nginx using node IP:nodeport. Here I assumed you have security group opening the node port.

Use this yaml. I updated the node port to be 31000. You can access the nginx on nodeport:31000. As I had mentioned you can not use 80 as it is for within the cluster. If you need to use 80, then you need ingress controller.

apiVersion: v1
kind: Service
metadata:
  name: testing-nodeport
  namespace: default
  labels:
    infrastructure: fargate
    app: testing-app
spec:
  type: NodePort
  selector:
    app: testing-app
  ports:
    - port: 80
      targetPort: testing-port
      protocol: TCP
      nodePort: 31000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: testing-deployment
  namespace: default
  labels:
    infrastructure: fargate
    app: testing-app
spec:
  replicas: 1
  selector:
    matchLabels:
      infrastructure: fargate
      app: testing-app
  template:
    metadata:
      labels:
        infrastructure: fargate
        app: testing-app
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - name: testing-port
          containerPort: 80
user_28970080

Okay after 16+ hours of debugging this I finally figured out what's going on. On fargate you can't set the security groups per node like you can with managed node groups. I was setting the security group rules in the "Additional security groups" settings. However, fargate apparently completely ignores those settings and ONLY uses the security group from your "Cluster security group" setting. So in the EKS UI I set the correct rules in the "Cluster security group" and I can now hit my pod directly on a fargate instance.

Big take away from this. Only use "Cluster security group" for fargate nodes.

user_50538860
Related