asyncio - re-raise exception from a task

user_487873 0
Viewed 818

I am using asyncio to do some TCP comms. I have a Receive() function that does read() in an infinite loop. This runs as a background task using asyncio.create_task(Receive()).

Now, if the connection is closed by the peer an exception (or could be any other exception) is raised which I catch in the Receive() function. However, I would like to re-raise that exception so the outer code can decide what to do (e.g. re-connect).

Since the exception is raised in a task, I do not know how to retrieve it.

I've tried to create an example of what I mean:

import asyncio

async def divide(x):
    try:
        return 1/x
    except Exception as e:
        print("Divide inner exception: ", e)
        raise   # Re-raise so main() can handle it

async def someFn():
    asyncio.create_task(divide(0))  # Exception is never retrieved
    # await divide(0) # This will raise two exceptions - the original in divide() and in main()

async def main():
    try:
        await someFn()
        # Do other things while someFn() runs
    except Exception as e:
        print("main exception: ", e)

asyncio.run(main())

How do I get the task exception in main()?

python-asyncio python
2 Answers

How do I get the task exception in main()?

You can make use of the fact that a task that raises an exception completes, and instead of letting the task run in the background, actually await its completion. This requires the code that creates tasks not to retain the task object returned by create_task() and either return it to the caller or store it to a shared set of tasks. (Libraries such as trio don't even allow blindly spawning a background task and instead require every task to be accompanied by a nursery that defines who handles its exceptions).

For example:

async def someFn():
    # set up a background task, but also return it to the caller
    t = asyncio.create_task(divide(0))
    return t

async def other_things():
    await asyncio.sleep(1)

async def main():
    try:
        task = await someFn()
        # await both the background task and other things, immediately
        # propagating an exception in either
        await asyncio.gather(task, other_things())
    except Exception as e:
        print("main exception: ", e)
user_16008980

Another option would be to use the add_done_callback and this allows some flexibility in not having to know beforehand where in code to handle the task (as is required if using asyncio.gather() option).

import asyncio

async def divide(x):
    try:
        return 1/x
    except Exception as e:
        print("Divide inner exception: ", e)
        raise   # Re-raise so callback can handle it 
                # (or don't handle exception here and allow callback to manage)

async def someFn():
    task = asyncio.create_task(divide(0))
    task.add_done_callback(task_cb)

def task_cb(task):
    try:
        task.result()
    except asyncio.CancelledError:
        pass  # Task cancellation should not be logged as an error.
    except Exception as e:
        print('Exception raised by task: ', e)

async def main():
    try:
        await someFn()
        # Do other things while someFn() runs
    except Exception as e:
        # Handle the exception in some way
        print("main exception: ", e)

asyncio.run(main())
user_4878730
Related