Is python dictionary async safe?

user_520692 0
Viewed 4121

I have created a dictionary in my Python application where I save the data and I have two tasks that run concurrently and get data from external APIs. Once they get the data, they update the dictionary - each with a different key in the dictionary.

I want to understand if the dictionary is async safe or do I need to put a lock when the dictionary is read/updated?

The tasks also read the last saved value each time.

my_data = {}
asyncio.create_task(call_func_one_coroutine)
asyncio.create_task(call_func_two_coroutine)

async def call_func_one_coroutine():
  data =  await goto_api_get_data()
  my_data['one'] = data + my_data['one']


async def call_func_two_coroutine():
  data =  await goto_api_another_get_data()
  my_data['two'] = data + my_data['two']
python-asyncio python
1 Answers

I want to understand if the dictionary is async safe or do I need to put a lock when the dictionary is read/updated?

Asyncio is based on cooperative multitasking, and can only switch tasks at an explicit await expression or at the async with and async for statements. Since update of a single dictionary can never involve awaiting (await must completes before the update begins), it is effectively atomic as far as async multitasking is concerned, and you don't need to lock it. This applies to all data structures accessed from async code.

To take another example where there is no problem:

# correct - there are no awaits between two accesses to the dict d
key = await key_queue.get()
if key in d:
    d[key] = calc_value(key)

An example where a dict modification would not be async-safe would involve multiple accesses to the dict separated by awaits. For example:

# incorrect, d[key] could appear while we're reading the value,
# in which case we'd clobber the existing key
if key not in d:
    d[key] = await read_value()

To correct it, you can either add another check after the await, or use an explicit lock:

# correct (1), using double check
if key not in d:
    value = await read_value()
    # Check again whether the key is vacant. Since there are no awaits
    # between this check and the update, the operation is atomic.
    if key not in d:
        d[key] = value

# correct (2), using a shared asyncio.Lock:
async with d_lock:
    # Single check is sufficient because the lock ensures that
    # no one can modify the dict while we're reading the value.
    if key not in d:
        d[key] = await read_value()
user_16008980
Related