Pycharm debugger when using multiprocessing pool

user_10774462 0
Viewed 1478

My problem is happening with this setup :

  • Pycharm 2020.3 pro
  • multiprocessing.Pool
  • Macbook pro 2020 (M1)
  • Conda python 3.8

And most of all, it happens when I use the debugger of pycharm.

It shows 8 times (number of processes) in the console :
Error loading: /Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd_attach_to_process/attach_x86_64.dylib

Every processes are executed. Results are correct. I can see them with htop command. So it's only a debugger failure and doesn't really impact code execution (correct me if I am wrong).

This is the kind of code I run :

def func(x):
    return x+10


if __name__ == '__main__':
    poo = Pool()
    x = [[i] for i in range(10)]
    res = poo.starmap(func, x)
    print(res)

I can ignore for now those massive printing in my console but it's not really convenient. If someone has an idea to get rid of those...

conda pycharm macos multiprocessing python
2 Answers

This sounds like something JetBrains devs will need to work out for the M1 (consider dropping them a bug report). In the meantime, I suspect you could disable it under the options:

PyCharm > Preferences > Build, Execution, Deployment > Python Debugger

and unchecking the box "Attach to subprocess automatically while debugging". See the pertinent docs for reference.

user_5709180

The solution for that error is modify the file /Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd_attach_to_process/linux_and_mac/compile_mac.sh and replace all code for the next one:

g++ -fPIC -D_REENTRANT -std=c++11 -arch arm64 -c -o attach_x86_64.o attach.cpp
g++ -dynamiclib -nostartfiles -arch arm64 -o attach_x86_64.dylib attach_x86_64.o -lc
rm attach_x86_64.o
mv attach_x86_64.dylib ../attach_x86_64.dylib

then you can run the sh script, which will replace the attach_x86_64.dylib file.

Note. this change will be lost if you update your pycharm.

user_182753190
Related