Run two functions concurrently in python

userSStJ 0
Viewed 63

I have the two functions below and I want them to run in parallel just like we use go routines in Go. Currently these do not run in parallel.

Please note that there is no async code in each of the functions, and I need to use returned data from both functions for later use.

Code:

import asyncio

async def func1(i: int) -> int:
    print("Initiated func1")
    while i <= 100000000:
        i += 1
    print("Finished func1")
    return 100

async def func2() -> (str, int):
    print("Initiated func2")
    return "hello world", 200

async def main():
    res = await asyncio.gather(*(func1(10), func2()))
    return res

if __name__ == "__main__":
    data1, data2 = asyncio.run(main())
    print(f"func1: {data1}, func2: {data2}")

Actual output:

Initiated func1
Finished func1
Initiated func2
func1: 100, func2: ('hello world', 200)

Expected output:

Initiated func1
Initiated func2
Finished func1
func1: 100, func2: ('hello world', 200)
coroutine parallel-processing python-asyncio asynchronous python
2 Answers

because, you declare func that is not awaitable.

# sync code block in func1: not awaitable
while i <= 100000000:
    i += 1

add asyncio.sleep(1) in func or use concurrent.futures.ProcessPoolExecuto

In [1]: import asyncio
   ...:
   ...: async def func1(i: int) -> int:
   ...:     print("Initiated func1")
   ...:     await asyncio.sleep(1)
   ...:     print("Finished func1")
   ...:     return 100
   ...:
   ...: async def func2() -> (str, int):
   ...:     print("Initiated func2")
   ...:     await asyncio.sleep(1)
   ...:     return "hello world", 200
   ...:
   ...: async def main():
   ...:     res = await asyncio.gather(*(func1(10), func2()))
   ...:     return res
   ...:
   ...:

In [2]: await main()
Initiated func1
Initiated func2
Finished func1
Out[2]: [100, ('hello world', 200)]

In [3]: from concurrent.futures import ProcessPoolExecutor

In [4]: def func1(i: int) -> int:
   ...:     print("Initiated func1")
   ...:     while i <= 100000000:
   ...:         i += 1
   ...:     print("Finished func1")
   ...:     return 100
   ...:
   ...: def func2() -> (str, int):
   ...:     print("Initiated func2")
   ...:     return "hello world", 200
   ...:

In [5]: def main():
   ...:     with ProcessPoolExecutor(2) as pool:
   ...:         res = [pool.submit(func1, 10), pool.submit(func2)]
   ...:         return [res[0].result(), res[1].result()]
   ...:

In [6]: main()
Initiated func1
Initiated func2
Finished func1
Out[6]: [100, ('hello world', 200)]
phi friday0

You misunderstand how concurrency with async-functions works. It's nothing like go-routines. The asynchronous execution still happens in one single thread. The event loop just may switch execution whenever it encounters an await. This means, if you aren't doing I/O-bound tasks in your async functions, there will be no concurrency at all.

Even Threading will not help you in Python, but this has been discussed extensively on this platform (search for "GIL"). You can only achieve meaningful concurrency with your particular task with Python using multiprocessing.

If I understand you correctly, you don't need to deal with async functions at all. You can just use the multiprocessing module for this.

Daniil Fajnberg0
Related