Using Locust with pre-defined requests per second

user_39381 0
Viewed 1929

Looking at load testing tools, I found Locust and immediately found it appealing as a Python guy, but I'm not entirely sure if I can achieve the following scenario with it...

I'm tasked with load testing a REST API with real-life traffic. I've extracted 5 minutes worth of GET traffic from production Apache logs, and the idea would be to run those same requests with the same time distribution (between 1 and 36 requests per second) using the load testing tool. To this end, I've built a Python dictionary where a relative timestamp (xx:xx, i.e. mins:secs) is the key, and a list of URL paths to request on that second is the value.

We have a dump of the production database from a certain point in time restored to our test environment, and the requests are from the next five minutes following the moment when the dump was created. Between test runs, I'm changing the parameters of how the REST API connects to the database, so the test runs need to be as identical as possible for the metrics to be comparable.

I've looked at custom load shapes in the Locust documentation, and it seems like they might work, but I'm not positive. Can the custom tick method implementation achieve this:

At 0 seconds, make a set of 4 requests.
At 1 second, make a set of 2 requests.
At 2 seconds, make a set of 12 requests.
...time passes, a set of predefined requests happens each second...
At 4 minutes 59 seconds, make a set of 27 requests.
At 5 minutes, make a set of 14 requests.

How would this map onto Locust's capabilities? It wouldn't matter how many users are spawned, all that matters is how many requests are made, and at what point in time.

I really like Locust for its ease of use and familiar syntax, but is it suitable for testing with a static, repeatable request load like this?

EDIT: as it seems this approach may not be achievable without great difficulty (or at all), I've come up with an alternate approach, different enough to warrant a separate question: Getting Locust to send a predefined distribution of requests per second

locust load-testing
2 Answers

In short, no. Locust is centered around running a certain number of users and those users being fairly independent from each other.

You can control the throughput for each user using wait_time = constant_throughput(x) (https://docs.locust.io/en/stable/writing-a-locustfile.html#wait-time-attribute). You can also check out locust-plugins's constant_total_ips that targets a global throughput (https://github.com/SvenskaSpel/locust-plugins/blob/master/examples/constant_total_ips_ex.py).

But if you want to do an exact number of requests at an exact time, I dont think there is anything readily available.

You could use python to "sync up" all the users before firing the requests. I dont have a nice solution for it, but this kinda works (you need to be running exactly 10 users though, otherwise you will run in to concurrency issues - pretty sure there are better solutions out there)

users_waiting = 0
lock = Semaphore(10)
...
    @task
    def t(self):
        global users_waiting
        with lock:
            users_waiting = users_waiting + 1 if users_waiting < 10 else 1
            while self.environment.runner and users_waiting < 10:
                time.sleep(0.1)
            # actually do your requests here
user_15919210

What you want is a custom Load Shape. You define what you want Locust do at any given moment in code. Example from the docs:

class MyCustomShape(LoadTestShape):
    time_limit = 600
    spawn_rate = 20

    def tick(self):
        run_time = self.get_run_time()

        if run_time < self.time_limit:
            # User count rounded to nearest hundred.
            user_count = round(run_time, -2)
            return (user_count, spawn_rate)

        return None

But in your case, instead of user_count = round(run_time, -2) you could pull the values out of your dictionary. If you make the user_count and spawn_rate be the same, the next effect will be roughly whatever your number of users every time (assuming you have enough workers to be able to spawn the difference every time). Adjusting the spawn_rate could adjust and smooth out the differences so there's less abrupt changes in users, if that's desired.

user_73556370
Related