Python queue stop when max reached

Luke Wardford 0
Viewed 22

In the code below, I'm putting number on a queue from a thread and retrieving and printing them from the main thread. It's supposed to print number from 0 to 99 but it's stops at 9. The max size of the queue is 10.

def fetch(queue):
    for i in range(100):
        queue.put(i)

def main():
    queue = Queue(maxsize=10)
    Thread(target=fetch, args=(queue,)).start()

    while not queue.empty():
        item = queue.get()
        print(item)

When I run this code I get:

0
1
2
3
4
5
6
7
8
9

The program doesn't stop, terminating it with ctl+c results:

^CException ignored in: <module 'threading' from '/usr/lib/python3.10/threading.py'>
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1560, in _shutdown
    lock.acquire()
KeyboardInterrupt:
queue python
1 Answers

The queue.empty() method is notoriously unreliable due to the nature of threading. You should use a sentinal value to mark the end of the queue:

from threading import Thread
from queue import Queue
from time import sleep

def fetch(queue):
    sleep(1)
    for i in range(100):
        queue.put(i)
    queue.put(None)     # None is a sentinal value

def sink1(queue):
    while True:
        item = queue.get()
        if item == None:
            break
        print(item)

def main():
    queue = Queue(maxsize=10)
    t=Thread(target=fetch, args=(queue,))
    t.start()
    sink1(queue)

main()
print('Done')

I tried your code and it seemed to work for me. I then added sleep(1) to the fetch() function and then the program just quits immediately since the main thread immediately sees an empty queue.

quamrana0
Related