Traefik: Load Balance Across Three Node Docker Swarm

user_4215616 0
Viewed 1176

I am working on setting up a three node Docker swarm for a web application I support. Initially, we have Traefik setup as a reverse proxy. Traefik and the web app both run on the same web server and the web server is in a single node docker swarm. We are trying to add two additional nodes for application stability.

At the moment, I'm simply trying to understand Traefik load balancing along with Docker Swarm. I am deploying a Traefik v1.7 stack and including the whoami application. The docker-compose file for this first past looks like:

version: "3.7"
services:
  traefik:
    image: traefik:1.7.10
    command:
      - "--logLevel=INFO"
      - "--defaultentrypoints=http"
      - "--entryPoints=Name:http Address::80"
      - "--api"
      - "--ping"
      - "--docker"
      - "--docker.swarmMode=true"
      - "--docker.watch=true"
      - "--docker.endpoint=unix:///var/run/docker.sock"
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /var/lib/docker/containers
        target: /var/lib/docker/containers
    ports:
      - 6080:8080
      - target: 80
        published: 80
        mode: host
    deploy:
      placement:
        constraints:
          - node.role == manager
    networks:
      shared_network:
        aliases:
          - traefik

  whoami:
    image: containous/whoami
    networks:
      - shared_network
    deploy:
      replicas: 3
      placement:
        constraints:
          - node.role == manager
      labels:
        - "traefik.port=80"
        - "traefik.docker.network=shared_network"
        - "traefik.enable=true"
        - "traefik.frontend.rule=PathPrefix:/whoami"
        - "traefik.backend=whoami"
        - "traefik.frontend.backend=whoami"

networks:
  shared_network:
    external: true

So as you can see, I have Traefik and three whoami services running on the manager node. With this, I can curl the the url http://***.***.***.***/whoami and receive three different responses

Response one:

# curl http://[Server IP Address]/whoami
Hostname: 6d559ddeee2f
IP: 127.0.0.1
IP: 10.0.1.18
IP: 172.18.0.7
RemoteAddr: 10.0.1.14:50320
GET /whoami HTTP/1.1
...

Response two:

# curl http://[Server IP Address]/whoami
Hostname: ade08aec0180
IP: 127.0.0.1
IP: 10.0.1.18
IP: 172.18.0.7
RemoteAddr: 10.0.1.14:50320
GET /whoami HTTP/1.1
...

Response three:

# curl http://[Server IP Address]/whoami
Hostname: fbcc6371383b
IP: 127.0.0.1
IP: 10.0.1.17
IP: 172.18.0.10
RemoteAddr: 10.0.1.14:55568
GET /whoami HTTP/1.1
...

However, if I change the docker-compose file and allow the replicas to be spread across the swarm, the behavior changes.

Modified portion of compose file

...
  whoami:
    image: containous/whoami
    networks:
      - shared_network
    deploy:
      replicas: 3
...

I get one good response back, but subsequent call return a Gateway Timeout error. I would expect each call to curl http://[Server IP Address]/whoami would result is a response from any one of the hosts in the swarm

How can I ensure Traefik and Docker are properly load balancing between all swarm nodes?

traefik docker
1 Answers

Apparently Traefik can't drain the connections during update (maybe it doesn't have access to healthchecks and swarm info?).

To achieve a zero-downtime rolling update you should delegate the load-balancing to docker swarm itself:

# docker-compose.yml

services:
  your_service:
    deploy:
      labels:
        - traefik.docker.lbswarm=true

From the docs:

Enables Swarm's inbuilt load balancer (only relevant in Swarm Mode).

If you enable this option, Traefik will use the virtual IP provided by docker swarm instead of the containers IPs. Which means that Traefik will not perform any kind of load balancing and will delegate this task to swarm.

Further info:

https://github.com/traefik/traefik/issues/41

https://github.com/traefik/traefik/issues/1480

user_19910890
Related