Why would spawning a process make computation run twice as fast?

user_2725810 0
Viewed 27

The following computation takes about 10.4 seconds on my Precision 5520:

import time
before = time.time()
sum = 0
for i in range(1, 100000000):
    sum += i
print(time.time() - before, sum)

On the same laptop, the following takes only 5.2 seconds:

import multiprocessing as mp
import time

def foo():
    before = time.time()
    sum = 0
    for i in range(1, 100000000):
        sum += i
    print(time.time() - before, sum)

mp.Process(target=foo).start()

This result is consistent. In fact, it holds (with a slightly smaller speed-up factor) even if I run cpu_count processes simultaneously.

So, why would spawning a process make computation run twice as fast?

python-multiprocessing python-multithreading multithreading performance
1 Answers

It is not the process that make the computation fast, it is running the computation in a function. This is because global variable accesses are slower than local variable accesses using the CPython interpreter. If you simply run foo() in the same process, then the computation time is also twice lower. Here is an example:

import time

def foo():
    before = time.time()
    sum = 0
    for i in range(1, 100000000):
        sum += i
    print(time.time() - before, sum)

# Slow (8.806 seconds)
before = time.time()
sum = 0
for i in range(1, 100000000):
    sum += i
print(time.time() - before, sum)

# Fast (4.362 seconds)
foo()

Note overwriting the built-in function sum is a bit dangerous as it may break codes using it and cause weird errors.

user_129395570
Related