How do I run Locust in a distributed Docker configuration?

user_132319 0
Viewed 682

I'm working on running Locust with multiple workers in a Fargate environment, but I wanted to see what it looked like in a simple distributed Docker setup. I took the below docker-compose.yml from the website and modified it so that everything would run on localhost. I can start Locust just fine with docker-compose up --scale worker=4, and four workers and master come up, but when I try to run a test via the Web UI, I get:

Attaching to locust-distributed-docker-master-1, locust-distributed-docker-worker-1, locust-distributed-docker-worker-2, locust-distributed-docker-worker-3, locust-distributed-docker-worker-4
locust-distributed-docker-worker-1  | [2021-11-17 19:01:19,719] be1b465ae5c7/INFO/locust.main: Starting Locust 2.5.0
locust-distributed-docker-master-1  | [2021-11-17 19:01:19,956] 8769b6dcd3ed/INFO/locust.main: Starting web interface at http://0.0.0.0:8089 (accepting connections from all network interfaces)
locust-distributed-docker-master-1  | [2021-11-17 19:01:20,016] 8769b6dcd3ed/INFO/locust.main: Starting Locust 2.5.0
locust-distributed-docker-worker-4  | [2021-11-17 19:01:20,144] bd481d228ef6/INFO/locust.main: Starting Locust 2.5.0
locust-distributed-docker-worker-3  | [2021-11-17 19:01:20,716] 26af3d44e1c9/INFO/locust.main: Starting Locust 2.5.0
locust-distributed-docker-worker-2  | [2021-11-17 19:01:21,122] d536c752bdee/INFO/locust.main: Starting Locust 2.5.0
locust-distributed-docker-master-1  | [2021-11-17 19:01:42,998] 8769b6dcd3ed/WARNING/locust.runners: You are running in distributed mode but have no worker servers connected. Please connect workers prior to swarming.

The whole point of this exercise is to watch the console to see how the workers interact with the master, nothing else.

docker-compose.yml:

version: '3'

services:
  master:
    image: locustio/locust
    ports:
     - "8089:8089"
    volumes:
      - ./:/mnt/locust
    command: -f /mnt/locust/locustfile.py --master

  worker:
    image: locustio/locust
    volumes:
      - ./:/mnt/locust
    command: -f /mnt/locust/locustfile.py --worker --master-host 127.0.0.1
locust docker-compose docker
1 Answers

To use the example in this way, with workers running on the same machine as the master, it's easiest to follow the example exactly and just not include the master IP address. The trouble is networking between Docker containers locally doesn't work the same as regular networking between hosts. You'll have to research the differences.

Locust workers will automatically discover the master if they run on the same host if you don't specify an IP address for the master. In the example docker-compose.yml --master-host master refers to the master service that will run by name, creating a networking bridge between the worker containers and the master container to allow the workers to automatically discover the master. When you actually deploy the workers, you may need a different setup that does specify a separate IP address to communicate with if they're run on separate hosts.

So just follow the example directly and have your workers command like this:

  worker:
    image: locustio/locust
    volumes:
      - ./:/mnt/locust
    command: -f /mnt/locust/locustfile.py --worker --master-host master

That should result in output like this:

% docker compose up --scale worker=4
[+] Running 5/0
 ⠿ Container docker-compose_master_1  Created                                                       0.0s
 ⠿ Container docker-compose_worker_2  Created                                                       0.0s
 ⠿ Container docker-compose_worker_3  Created                                                       0.0s
 ⠿ Container docker-compose_worker_1  Created                                                       0.0s
 ⠿ Container docker-compose_worker_4  Created                                                       0.0s
Attaching to master_1, worker_1, worker_2, worker_3, worker_4
worker_3  | [2021-11-18 16:32:49,911] d90df67c6a69/INFO/locust.main: Starting Locust 2.5.0
worker_4  | [2021-11-18 16:32:50,062] 112a60412b1e/INFO/locust.main: Starting Locust 2.5.0
master_1  | [2021-11-18 16:32:50,224] 859d07f8570b/INFO/locust.main: Starting web interface at http://0.0.0.0:8089 (accepting connections from all network interfaces)
worker_2  | [2021-11-18 16:32:50,233] 56ffce9d4448/INFO/locust.main: Starting Locust 2.5.0
master_1  | [2021-11-18 16:32:50,238] 859d07f8570b/INFO/locust.main: Starting Locust 2.5.0
master_1  | [2021-11-18 16:32:50,239] 859d07f8570b/INFO/locust.runners: Client '56ffce9d4448_dfda9f3bcff742909af80b63d7866714' reported as ready. Currently 1 clients ready to swarm.
master_1  | [2021-11-18 16:32:50,249] 859d07f8570b/INFO/locust.runners: Client '112a60412b1e_49c9e2df265d4fd7bc0f6554a76e66c9' reported as ready. Currently 2 clients ready to swarm.
worker_1  | [2021-11-18 16:32:50,256] 988707b23133/INFO/locust.main: Starting Locust 2.5.0
master_1  | [2021-11-18 16:32:50,259] 859d07f8570b/INFO/locust.runners: Client '988707b23133_88ac7446afd843a5ae7a20dceaed9ea4' reported as ready. Currently 3 clients ready to swarm.
master_1  | [2021-11-18 16:32:50,336] 859d07f8570b/INFO/locust.runners: Client 'd90df67c6a69_e432779d02f94947abb992ff1043eb0e' reported as ready. Currently 4 clients ready to swarm.
user_73556370
Related