async exec in python

user_1984065 0
Viewed 7823

I'd like to call exec in an async function and do something like the following code (which is not valid):

import asyncio

async def f():
    await exec('x = 1\n' 'await asyncio.sleep(x)')

More precisely, I'd like to be able to wait for a future inside the code that runs in exec.

How can this be achieved?

python-asyncio exec python-3.x python
7 Answers

Note: F-strings are only supported in python 3.6+. For older versions, use %s, .format() or the classic + concatenation.

async def aexec(code):
    # Make an async function with the code and `exec` it
    exec(
        f'async def __ex(): ' +
        ''.join(f'\n {l}' for l in code.split('\n'))
    )

    # Get `__ex` from local variables, call it and return the result
    return await locals()['__ex']()

Known issues:

  • If you use new lines in a string (triple quotes), it will mess up the formatting.
user_96218130

This is based off @YouTwitFace's answer, but keeps globals unchanged, handles locals better and passes kwargs. Note multi-line strings still won't keep their formatting. Perhaps you want this?

async def aexec(code, **kwargs):
    # Don't clutter locals
    locs = {}
    # Restore globals later
    globs = globals().copy()
    args = ", ".join(list(kwargs.keys()))
    exec(f"async def func({args}):\n    " + code.replace("\n", "\n    "), {}, locs)
    # Don't expect it to return from the coro.
    result = await locs["func"](**kwargs)
    try:
        globals().clear()
        # Inconsistent state
    finally:
        globals().update(**globs)
    return result

It starts by saving the locals. It declares the function, but with a restricted local namespace so it doesn't touch the stuff declared in the aexec helper. The function is named func and we access the locs dict, containing the result of the exec's locals. The locs["func"] is what we want to execute, so we call it with **kwargs from aexec invocation, which moves these args into the local namespace. Then we await this and store it as result. Finally, we restore locals and return the result.

Warning:

Do not use this if there is any multi-threaded code touching global variables. Go for @YouTwitFace's answer which is simpler and thread-safe, or remove the globals save/restore code

user_55095750

Here is a more robust way using the builtin ast module:

import ast

async def async_exec(stmts, env=None):
    parsed_stmts = ast.parse(stmts)

    fn_name = "_async_exec_f"

    fn = f"async def {fn_name}(): pass"
    parsed_fn = ast.parse(fn)

    for node in parsed_stmts.body:
        ast.increment_lineno(node)

    parsed_fn.body[0].body = parsed_stmts.body
    exec(compile(parsed_fn, filename="<ast>", mode="exec"), env)

    return await eval(f"{fn_name}()", env)
user_6405840

Here's a module using AST to do stuff. This means that mutli-line strings will work perfectly and line-numbers will match the original statements. Also, if anything is an expression, it is returned (as a list if there are multiple, otherwise as just an element)

I made this module (check the revision history of this answer for more details on the inner workings). I use it here

user_55095750

Just use this function:

import asyncio
async def async_exec(code):
    t = [None]
    exec('async def _async_exec():\n return {}\nt[0] = asyncio.ensure_future(_async_exec())'.format(code))
    return await t[0]

Here is a code example that can directly run. (It works for Python 3.6.8)

import asyncio
async def async_exec(code):
    t = [None]
    exec('async def _async_exec():\n return {}\nt[0] = asyncio.ensure_future(_async_exec())'.format(code))
    return await t[0]

async def p(s):
    await asyncio.sleep(s)
    return s


async def main():
    print(await async_exec('await p(0.1) / await p(0.2)'))


asyncio.get_event_loop().run_until_complete(main())

I try to explain it, define an async function in exec. Inside the async function, run the code you want. But exec doesn't have return value, use t[0] to store an asyncio future, await the future outside of exec to get the return value.

user_101963940
Related