asyncio.gather with generator expression

user_2606953 0
Viewed 1954

Why doesn't asyncio.gather work with a generator expression?

import asyncio

async def func():
    await asyncio.sleep(2)

# Works
async def call3():
    x = (func() for x in range(3))
    await asyncio.gather(*x)

# Doesn't work
async def call3():
    await asyncio.gather(func() for x in range(3))

# Works
async def call3():
    await asyncio.gather(*[func() for x in range(3)])

asyncio.run(call3())

The second variant gives:

[...]
  File "test.py", line 13, in <genexpr>
    await asyncio.gather(func() for x in range(3))
RuntimeError: Task got bad yield: <coroutine object func at 0x10421dc20>

Is this expected behavior?

python-asyncio python
3 Answers
await asyncio.gather(func() for x in range(3))

This doesn't work because this is passing the generator object as argument to gather. gather doesn't expect an iterable, it expects coroutines as individual arguments. Which means you need to unpack the generator.

user_4760

Unpack the generator:

await asyncio.gather(*(func() for i in range(10)))  # star expands generator

We must expand it because asyncio.gather expects a list of arguments (i.e. asyncio.gather(coroutine0, coroutine1, coroutine2, coroutine3)), not an iterable

user_41092490

Python uses */** for both 'un-packing' and just 'packing' based on whether it's used for variable assignment or not.

def foo(*args,**kwargs):...

In this case, all non-keyworded args are getting put into a tuple args and all kwargs are getting packed into a new dictionary. A single variable passed in still gets packed into a tuple(*) or dict(**).

This is kind of a hybrid

first,*i_take_the_rest,last = range(10)
>>> first=0,i_take_the_rest=[1,2,3,4,5,6,7,8],last=9
*a,b = range(1)
>>> a=[],b=0

But here it unpacks:

combined_iterables = [*range(10),*range(3)]
merged_dict = {**first_dict,**second_dict}

So basically if it's on the left side of the equals or if it's used in a function/method definition like *foo it's packing stuff into a list or tuple (respectively). In comprehensions, however, it has the unpacking behavior.

user_72620490
Related