Python Threading Self Calling Threads Unexpected Behavior

Question

I am testing a method to run several tasks in parallel. These tasks will run in parallel threads and I want the tasks to repeat until a global variable is set. I am first trying threading to launch the parallel threads, and make sure they will work properly. What I have so far:

import threading
from IPython.display import clear_output
import time

i = 0
j = 0
def main():
    global i
    global j
    t1 = threading.Thread(name = "task1", target = task1)
    t2 = threading.Thread(name = "task2", target = task2)
    t1.start()
    t2.start()

def task1():
    global i
    i += 1
    time.sleep(10)
    t1 = threading.Thread(name = "task1", target = task1)
    t1.start()

def task2():
    global j
    j -= 1
    time.sleep(10)
    t2 = threading.Thread(name = "task2", target = task2)
    t2.start()

tmain = threading.Thread(name = "main", target = main)
tmain.start()

which starts a main thread that then starts two threads which run task1 and task2. To monitor the current threads and the values of i and j I run:

while(True):
    clear_output(wait=True)
    for thread in threading.enumerate():
        print(thread)
    print(i)
    print(j)
    time.sleep(0.1)

(all of this is being run in a Jupyter Notebook).

Running the script above, i noticed some unexpected results. I expect that at any given time, there should be at most two threads of task1 and task2, but instead I observe many more threads of task2 compared to task1. These are not ghost or finished threads, because the absolute values of i and j grow disproportionately. Two observations I made:

Again, i expect that there should be a symmetric number of threads for both task1 and task 2, and I also expect that the abslute values of i and j should grow more proportionately than they are. Any insight on how to mitigate this discrepancy or avoid this issue would be appreciated.

user_11111990 · Answer

I ran your code in Jupyter and didn't have your problem.

<_MainThread(MainThread, started 139735228168000)>











272
-272

But as you already saw with your own code, there are multiple instances of each task running. So after a task 'has started itself anew' it takes some time before it kills itself.

A solution to your Jupyter problem could be to give the main function the control of restarting a killed tasked. This ensures that always only 1 thread of each task is running.

import threading
from IPython.display import clear_output
import time

i = 0
j = 0

main_stop = False


def task1():
    global i
    i += 1
    time.sleep(4)


def task2():
    global j
    j -= 1
    time.sleep(4)


def main():
    global i
    global j
    t1 = threading.Thread(name="task1", target=task1)
    t2 = threading.Thread(name="task2", target=task2)
    t1.start()
    t2.start()
    while not main_stop:
        if not t1.is_alive():
            del t1
            t1 = threading.Thread(name="task1", target=task1)
            t1.start()
        if not t2.is_alive():
            del t2
            t2 = threading.Thread(name="task2", target=task2)
            t2.start()

    # wait for tasks to complete
    while t1.is_alive():
        time.sleep(0.1)
    while t2.is_alive():
        time.sleep(0.1)


tmain = threading.Thread(name="main", target=main)
tmain.start()

run_time = 30   # seconds
end_time = time.time() + run_time
while time.time() < end_time:

    clear_output(wait=True)
    for thread in threading.enumerate():
        print(thread)
    print(i)
    print(j)
    time.sleep(0.1)

main_stop = True
# wait for main to complete
while tmain.is_alive():
    time.sleep(0.1)

print('program completed')

Python Threading Self Calling Threads Unexpected Behavior

1 Answers