My co-worker asked me why his code cannot run in concurrency in multi-threads. I discover the os.system function is acting weird unlike the other functions in multi-threads. I code a small demo to reproduce the problem. Appreciating somebody who can answer my question.
Run like serial execution
import os
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
executor = ThreadPoolExecutor()
begin = time.time()
for _ in as_completed([
executor.submit(os.system, cmd)
for cmd in ['sleep 10', 'sleep 3', 'sleep 2']
]):
print("completed after", time.time() - begin, "seconds")
output
completed after 10.005892276763916 seconds
completed after 13.017478942871094 seconds
completed after 15.034425258636475 seconds
Run normally
Replace os.system with subprocess.call.
import subprocess
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
executor = ThreadPoolExecutor()
begin = time.time()
for _ in as_completed([
executor.submit(subprocess.call, cmd, shell=True)
for cmd in ['sleep 10', 'sleep 3', 'sleep 2']
]):
print("completed after", time.time() - begin, "seconds")
output
completed after 2.019604206085205 seconds
completed after 3.017033338546753 seconds
completed after 10.009897232055664 seconds