Queue of dicts only

user_13104510 0
Viewed 48

I'm writing a socket server that passes the server state to the client when requested. The state is passed as a JSON but in the server the data is stored in a dictionary. The socketserver handler pulls the server state from a queue (the server runs in a thread) when the client requests the server state.

My problem is that I always get only the final queue entry when I get from the queue. I can see the queue size decrementing as I pull entries from the queue. Here's a very simple script that illustrates the issue:

import queue

appState = {"console" : {"message" : "NA"}}

appStateQueue = queue.Queue()

print("Putting dict data on the queue:")
for i in range(10):
        appState["console"]["message"] = str(i)
        print(appState) 
        appStateQueue.put_nowait(appState)

print("Queue size:", appStateQueue.qsize())

print("Getting dict data from the queue:")
for i in range(10):
        print(appStateQueue.get(), appStateQueue.qsize())

print("Putting integer data on the queue:")
for i in range(10):
        appState = i
        print(appState)
        appStateQueue.put_nowait(appState)

print("Queue size:", appStateQueue.qsize())

print("Getting integer data from the queue:")
for i in range(10):
        print(appStateQueue.get(), appStateQueue.qsize())

The output is as follows:

Putting dict data on the queue:
{'console': {'message': '0'}}
{'console': {'message': '1'}}
{'console': {'message': '2'}}
{'console': {'message': '3'}}
{'console': {'message': '4'}}
{'console': {'message': '5'}}
{'console': {'message': '6'}}
{'console': {'message': '7'}}
{'console': {'message': '8'}}
{'console': {'message': '9'}}
Queue size: 10
Getting dict data from the queue:
{'console': {'message': '9'}} 9
{'console': {'message': '9'}} 8
{'console': {'message': '9'}} 7
{'console': {'message': '9'}} 6
{'console': {'message': '9'}} 5
{'console': {'message': '9'}} 4
{'console': {'message': '9'}} 3
{'console': {'message': '9'}} 2
{'console': {'message': '9'}} 1
{'console': {'message': '9'}} 0
Putting integer data on the queue:
0
1
2
3
4
5
6
7
8
9
Queue size: 10
Getting integer data from the queue:
0 9
1 8
2 7
3 6
4 5
5 4
6 3
7 2
8 1
9 0

Notice how the dict data from the queue is always that same. It's always the last data put on the queue. The integer data behaves as I'd expect. Strings do as well, although I don't show it here.

What's going on here?

queue python-3.x dictionary
1 Answers

As, I see you are not creating new dicts and only changing the value of an existing dict variable. This can be easily understood by the below snippet of code,

>>> appState = {"console" : {"message" : "NA"}}
>>> appState
{'console': {'message': 'NA'}}

>>> appState_copy = appState
>>> appState_copy["console"]["message"] = 23

>>> appState
{'console': {'message': 23}}
>>> appState_copy
{'console': {'message': 23}}

Python only copies the reference when you assign a dict to a new variable, so when you are assigning an integer in every iteration, you are essentially changing the value of the same dict and not creating a new one.

since you are using a nested dict, you need to make use deepcopy method from copy module to make this work.

Here is a working solution:

import queue
import copy #added import statememt

appState = {"console" : {"message" : "NA"}}

appStateQueue = queue.Queue()

print("Putting dict data on the queue:")
for i in range(10):
        new_appState = copy.deepcopy(appState) #creating a copy of dict every iteration
        new_appState["console"]["message"] = str(i)
        print(new_appState) 
        appStateQueue.put_nowait(new_appState) #putting the created copy in the queue

print("Queue size:", appStateQueue.qsize())

print("Getting dict data from the queue:")
for i in range(10):
        print(appStateQueue.get(), appStateQueue.qsize())

print("Putting integer data on the queue:")
for i in range(10):
        appState = i
        print(appState)
        appStateQueue.put_nowait(appState)

print("Queue size:", appStateQueue.qsize())

print("Getting integer data from the queue:")
for i in range(10):
        print(appStateQueue.get(), appStateQueue.qsize())
user_15938480
Related