nginx: [emerg] host not found in upstream "udagram-users:8080" in /etc/nginx/nginx.conf:11

user_13557906 0
Viewed 1180

After deploying to AWS EKS I get this error

Gtihub repo: https://github.com/oussamabouchikhi/udagram-microservices


Steps to reproduce

  1. Create AWS EKS cluster and node groups
  2. Configure EKS cluster with kubectl
  3. Deploy to EKS cluster (secrets first, then other services, then reverserproxy)
    . kubectl apply -f env-secret.yaml
    . kubectl apply -f aws-secret.yaml
    . kubectl apply -f env-configmap.yaml
    . ...
    . kubectl apply -f reverseproxy-deployment.yaml
    . kubectl apply -f reverseproxy-service.yaml

enter image description here

nginx-config

worker_processes 1;
  
events { worker_connections 1024; }
error_log /dev/stdout debug;

http {

    sendfile on;

    upstream users {
        server udagram-users:8080;
    }

    upstream feed {
        server udagram-feed:8080;
    }
    
    proxy_set_header   Host $host;
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Host $server_name;
    
    server {
        listen 8080;
        location /api/v0/feed {
            resolver           8.8.8.8;
            proxy_pass         http://feed;
        }
        location /api/v0/users {
            resolver           8.8.8.8;
            proxy_pass         http://users;
        }            
    }

}

docker-compose

version: '3'
services:
  reverseproxy:
    image: oussamabouchikhi/reverseproxy
    ports:
      - 8080:8080
    restart: always
    depends_on:
      - udagram-users
      - udagram-feed
    networks:
      - example-net
  udagram-users:
    image: oussamabouchikhi/udagram-api-users
    volumes:
      - $HOME/.aws:/root/.aws
    environment:
      POSTGRES_USERNAME: $POSTGRES_USERNAME
      POSTGRES_PASSWORD: $POSTGRES_PASSWORD
      POSTGRES_DB: $POSTGRES_DB
      POSTGRES_HOST: $POSTGRES_HOST
      AWS_REGION: $AWS_REGION
      AWS_PROFILE: $AWS_PROFILE
      AWS_MEDIA_BUCKET: $AWS_BUCKET
      JWT_SECRET: $JWT_SECRET
      URL: $URL
      networks:
        - example-net
  udagram-feed:
    image: oussamabouchikhi/udagram-api-feed
    volumes:
      - $HOME/.aws:/root/.aws
    environment:
      POSTGRES_USERNAME: $POSTGRES_USERNAME
      POSTGRES_PASSWORD: $POSTGRES_PASSWORD
      POSTGRES_DB: $POSTGRES_DB
      POSTGRES_HOST: $POSTGRES_HOST
      AWS_REGION: $AWS_REGION
      AWS_PROFILE: $AWS_PROFILE
      AWS_MEDIA_BUCKET: $AWS_BUCKET
      JWT_SECRET: $JWT_SECRET
      URL: $URL
    networks:
      - example-net
  udagram-frontend:
    image: oussamabouchikhi/udagram-frontend
    ports:
      - '8100:80'
    networks:
      - example-net
networks:
  example-net:
    external: true

reverseproxy-deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    service: reverseproxy
  name: reverseproxy
spec:
  replicas: 1
  selector:
    matchLabels:
      service: reverseproxy
  template:
    metadata:
      labels:
        service: reverseproxy
    spec:
      containers:
        - image: oussamabouchikhi/reverseproxy:latest
          name: reverseproxy
          imagePullPolicy: Always
          resources:
            requests:
              memory: '64Mi'
              cpu: '250m'
            limits:
              memory: '1024Mi'
              cpu: '500m'
          ports:
            - containerPort: 8080
      restartPolicy: Always

reverseproxy-service

apiVersion: v1
kind: Service
metadata:
  labels:
    service: reverseproxy
  name: reverseproxy
spec:
  ports:
  - name: "8080"
    port: 8080
    targetPort: 8080
  selector:
    service: reverseproxy
  type: LoadBalancer
nginx-reverse-proxy amazon-eks kubernetes amazon-web-services nginx
1 Answers

Use resolver in nginx config

The nginx resolver directive is required.

Nginx is a multiplexing server (many connections in one OS process), so each call of system resolver will stop processing all connections till the resolver answer is received. That's why Nginx implemented its own internal non-blocking resolver.

If your config file has static DNS names (not generated), and you do not care about track IP changes without nginx reload, you don't need nginx's resolver. In this case all DNS names will be resolved on startup. Nginx's resolver

Nginx resolver directive should be used, if you want to resolve domain name in runtime without nginx reload.

E.g.:

location /my_uri {
  resolver kube-dns.kube-system valid=10s;
  ...
}

location /my_uri {
  resolver 127.0.0.1:53 ipv6=off valid=10s;
  ...
}

Use the same network (not your case, but still worth noting)

Containers you are trying to link may not be on the same network. You may want to put them all on the same network.

In your case subnets are the same, it's ok:

docker-compose

version: '3'
services:
  reverseproxy:
    ...
    networks:
      - example-net
  udagram-users:
    ...
      networks:
        - example-net
  udagram-feed:
    ...
    networks:
      - example-net
  udagram-frontend:
    ...
    networks:
      - example-net
networks:
  example-net:
    external: true
user_57208180
Related