Asynchronous execution of Python subprocess.Popen with wait()

user_10061137 0
Viewed 2769

I'm new to Python async concepts. However I want to sequentially call 2 functions, the order is what matters here:

def func1():
    p = subprocess.Popen('ffmpeg -i in.webm out.mp4')
    p.wait()

def func2():
    os.remove('in.webm')

func1()
func2()

Function 1 contains subprocess which converts video files, and function 2 deletes the input afterwards.
Popen with p.wait() does force synchronous execution but it comes at the price of blocking the main thread.
Popen without p.wait() doesn't block it but makes function 2 to get called before ffmpeg will complete its job.

I would like to achieve something similar to Javascript promises or async/await constructs like this:

async func1() {
    let response = await fetch('/api');
    let user = await response.json();
    return user;
}

func2(func1());

Sounds like a trivial task but I'm lost between multithreading, multiprocessing and asyncio.
How can I make function 2 to await for the output of function 1 without blocking the main thread?

python-asyncio python-multithreading popen subprocess python
1 Answers

To spawn subprocesses in an asyncio program, use asyncio's subprocess primitives which provide an interface similar to that of subprocess, but exposing awaitable coroutines.

The equivalent of your JavaScript example would be something like this:

async def main():
    # Schedule func1() to run, but don't await it. (In JavaScript
    # it would be enough to just call func1().)
    asyncio.create_task(func1())

    print('we get here immediately')

    # Await something to keep the event loop spinning - without
    # this, the program would exit at this point. In real code
    # this would await an exit signal, server.wait_closed(), etc.
    await asyncio.Event().wait()

async def func1():
    # Like in JavaScript, await suspends the current function
    # until a result is available.
    res = await fetch('/api1', {})
    userid = (await res.json()).id
    asyncio.create_task(func2(userid))

async def func2(userid):
    res = await fetch('/api2', ...userid...)
    func3((await res.json()).url)

def func3(url):
    print('user photo', url)

asyncio.run(main())
user_16008980
Related