How to detect data races in Python applications?

user_13859012 0
Viewed 734

It's not really clear what tools can be used to detect possible occurences of data races in python scripts, python code, or python-based executables.

I know that at least for CPython, only one thread is allowed to be executed to the GIL. But that doesn't mean data races cannot occur (for instance when two threads write an attribute on a structure, each with a value).

So how can I detect such scenarios?

python-multithreading race-condition python
1 Answers

Assuming you've already done the obvious first step and looked through the code for non-atomic operations like v += 1 on shared variables...

You can expand the window for race conditions to show up by increasing the frequency of a thread dropping the GIL. You can do so by lowering the switch-interval from its default of 5 ms. You can also insert time.sleep(0) around suspicious areas in the code to let a thread drop the GIL at this place if another one is trying to acquire it at that moment. Also randomizing the inserted sleep times ("fuzzing") is an option to further skew the execution order of threads.

Because it might not be clear to everyone what the point of doing this is:

Your code might work because a thread can comfortably walk through critical sections in it's 5 ms standard time slice and you get "lucky" with OS-scheduling during testing. Later you add some code and the critical sections suddenly is on the edge of the time slice and you get to note the race condition through weird bugs.

Demo code for letting race condition show up with help of changing sys.setswitchinterval:

from threading import Thread
import sys
# import time


def foo(n):
    global global_v
    for _ in range(n):
        x = global_v
        for _ in range(100):  # expand race condition for demo
            pass
        # time.sleep(0) # also: multiple locations with randomizing sleep times
        global_v = x + 1


def run(n_workers, n_iter):
    print(f"test with switch-interval: {sys.getswitchinterval()} sec")
    pool = [Thread(target=foo, args=(n_iter,)) for _ in range(n_workers)]

    for t in pool:
        t.start()
    for t in pool:
        t.join()

    print(f"expected: {n_workers * n_iter}, actual: {global_v}")


if __name__ == '__main__':

    N_WORKERS = 8
    n_iter = 1000
    global_v = 0

    # sys.setswitchinterval(0.001)
    run(N_WORKERS, n_iter)

Output:

test with switch-interval: 0.005 sec
expected: 8000, actual: 8000

Process finished with exit code 0

Output with sys.setswitchinterval(0.001):

test with switch-interval: 0.001 sec
expected: 8000, actual: 7318

Process finished with exit code 0
user_90594200
Related