multiprocessing inside async in python 3.8 fails (Can't pickle local object)

user_8396941 0
Viewed 515

The following code does work in python 3.7, but it doesn't work in python 3.8 (AttributeError: Can't pickle local object 'main.<locals>.f')

import multiprocessing as mp
import asyncio

async def main():
    def f():
        print("hello")

    p = mp.Process(target=f)
    p.start()

if __name__ == "__main__":
    asyncio.run(main())

I know that running a process inside an async function is not a common thing and might raise a few eyebrows, but I've found it useful sometimes. What is the reason it doesn't work in python 3.8 anymore? Is there a way to update the code to make it work?

python-3.8 python-asyncio multiprocessing python
1 Answers

I just realized that this has nothing to do with async. It fails without async too. The fix was to move f outside of main. Not sure why this is now required in python 3.8 though.

import multiprocessing as mp
import asyncio

def f():
    print("hello")

async def main():
    p = mp.Process(target=f)
    p.start()

if __name__ == "__main__":
    asyncio.run(main())
user_83969410
Related