Not possible to chain native asyncio coroutines by simply returning them

user_204760 0
Viewed 441

I've been using py3.4's generator-based coroutines and in several places I've chained them by simply having one coroutine call return inner_coroutine() (like in the example below). However, I'm now converting them to use py3.5's native coroutines and I've found that no longer works as the inner coroutine doesn't get to run (see output from running the example below). In order for the native inner coroutine to run I need to use a return await inner_coroutine() instead of the original return inner_coroutine().

I expected chaining of native coroutines to work in the same way as the generator-based ones, and can't find any documentation stating otherwise. Am I missing something or is this an actual limitation of native coroutines?

import asyncio

@asyncio.coroutine
def coro():
    print("Inside coro")

@asyncio.coroutine
def outer_coro():
    print("Inside outer_coro")
    return coro()

async def native_coro():
    print("Inside native_coro")

async def native_outer_coro():
    print("Inside native_outer_coro")
    # return await native_coro()  # this works!
    return native_coro()

loop = asyncio.get_event_loop()
loop.run_until_complete(outer_coro())
loop.run_until_complete(native_outer_coro())

And the output from running that example:

Inside outer_coro
Inside coro
Inside native_outer_coro
foo.py:26: RuntimeWarning: coroutine 'native_coro' was never awaited
loop.run_until_complete(native_outer_coro())
python-asyncio python-3.x python
2 Answers

This is the same content as another answer, but stated in a way that I think will be easier to understand as a response to the question.

The way python determines whether something is a generator or a normal function is whether it contains a yield statement. This creates an ambiguity with @asyncio.coroutine. Whether your coroutine executes immediately or whether it waits until the caller calls next on the resulting generator object depends on whether your code actually happens to include a yield statement. The native coroutines are by design unambiguously generators even if they do not happen to include any await statements. This provides predictable behavior, but does not permit the form of chaining you are using. You can as you pointed out do

    return await inner_coroutine()

However note that in that await syntax, the inner coroutine is called while executing the outer coroutine in the event loop. However, with the generator-based approach and no yield, the inner coroutine is constructed while actually submitting the coroutine to the event loop. In most circumstances this difference does not matter.

user_25948120

Your old version had wrong logic and worked only due to imperfect generator-based implementation. New syntax allowed to close this feature and make asyncio more consistent.

Idea of coroutines is to work like this:

c = coro_func()     # create coroutine object
coro_res = await c  # await this object to get result

In this example...

@asyncio.coroutine
def outer():
    return inner()

...awaiting of outer() should return inner() coroutine object not this object's result. But due to imperfect implementation it awaits of inner() (like if yield from inner() was written).

In new syntax asyncio works exactly as it should: it returns coroutine object instead of it's result. And since this coroutine object was never awaited (what usually means mistake) you get this warning.

You can change your code like this to see it all clearly:

loop = asyncio.get_event_loop()
print('old res:', loop.run_until_complete(outer_coro()))
print('new res:', loop.run_until_complete(native_outer_coro()))
user_11132070
Related