First of all, I've only started working with Python a month ago, so I don't have deep knowledge about anything.
In a project I'm trying to collect the results of multiple (simultaneous) functions in a database, infinitely until I tell it to stop. In an earlier attempt, I successfully used multiprocessing to do what I need, but since I now need to collect all the results of those functions for a database within the main, I switched to threading instead.
Basically, what I'm trying to do is:
collect1 = Thread(target=collect_data1)
collect2 = Thread(target=collect_data2)
send1 = Thread(target=send_data1)
send2 = Thread(target=send_data2)
collect = (collect1, collect2)
send = (send1, send2)
while True:
try:
for thread in collect:
thread.start()
for thread in collect:
thread.join()
for thread in send:
thread.start()
for thread in send:
thread.join()
except KeyboardInterrupt:
break
Now, obviously I can't just restart the threads. Nor explicitly kill them. The functions within the threads can theoretically be stopped at any point, so terminate() from multiprocessing was fine.
I was thinking whether something like the following could work (at least PyCharm is fine with it, so it seems to work) or if it creates a memory leak (which I assume), because the threads are never properly closed or deleted, at least as far as I can tell from researching. Again, I'm new to Python, so I don't know anything about this aspect.
Code:
while True:
try:
collect1 = Thread(target=collect_data1)
collect2 = Thread(target=collect_data2)
send1 = Thread(target=send_data1)
send2 = Thread(target=send_data2)
collect = (collect1, collect2)
send = (send1, send2)
for thread in collect:
thread.start()
for thread in collect:
thread.join()
for thread in send:
thread.start()
for thread in send:
thread.join()
except KeyboardInterrupt:
break
I feel like this approach seems too good to be true, especially since I've never came across a similar solution during my research.
Anyways, any input is appreciated.
Have a good day.