How to increase RPS in distributed locust load test

Viewed 5711

I cannot get past 1200 RPS no matter if I use 4 or 5 workers.

I tried to start locust in 3 variations -- one, four, and five worker processes (docker-compose up --scale worker_locust=num_of_workers). I use 3000 clients with a hatch rate of 100. The service that I am loading is a dummy that just always returns yo and HTTP 200, i.e., it's not doing anything, but returning a constant string. When I have one worker I get up to 600 RPS (and start to see some HTTP errors), when I have 4 workers I can get up to the ~1200 RPS (without a single HTTP error):

enter image description here

When I have 5 workers I get the same ~1200 RPS, but with a lower CPU usage: enter image description here

I suppose that if the CPU went down in the 5-worker case (with respect to 4-worker case), than it's not the CPU that is bounding the RPS.

I am running this on a 6-core MacBook.

The locustfile.py I use posts essentially almost empty requests (just a few parameters):

from locust import HttpUser, task, between, constant


class QuickstartUser(HttpUser):
    wait_time = constant(1)  # seconds

    @task
    def add_empty_model(self):
        self.client.post(
            "/models",
            json={
                "grouping": {
                    "grouping": "a/b"
                },
                "container_image": "myrepo.com",
                "container_tag": "0.3.0",
                "prediction_type": "prediction_type",
                "model_state_base64": "bXkgc3RhdGU=",
                "model_config": {},
                "meta": {}
            }
        )

My docker-compose.yml:

services:
  myservice:
      build:
          context: ../
      ports:
          - "8000:8000"

  master_locust:
      image: locustio/locust
      ports:
          - "8089:8089"
      volumes:
          - ./:/mnt/locust
      command: -f /mnt/locust/locustfile.py --master

  worker_locust:
      image: locustio/locust
      volumes:
          - ./:/mnt/locust
      command: -f /mnt/locust/locustfile.py --worker --master-host master_locust

Can someone suggest the direction of getting towards the 2000 RPS?

1 Answers

You should check out the FAQ.

https://github.com/locustio/locust/wiki/FAQ#increase-my-request-raterps

It's probably your server not being able to handle more requests, at least from your one machine. There are other things you can do to make more sure that's the case. You can try FastHttpUser, running on multiple machines, or just upping the number of users. But if you can, check to see how the server is handling the load and see what you can optimize there.

Related