I have developed a deep-learning object-detect program based on pytorch and it works very well. Today I deploy this program on a PC, everything goes well, but the program cannot be launched. Debug and find out that, the program blocks, or halts, on importing pytorch.
Simply start a python prompt, type import torch and the prompt blocks. top command shows that CPU/memory usage is very low. Press ctrl-c cannot stop the prompt. While other library importing is fine. I have tried pycrypto and the one I wrote myself, all work but pytorch cannot.
I have deployed more than 100 times, but never meet this situation. I also tried to reinstall pytorch, from 1.6 to 1.4, torchvision from 0.7 to 0.5, still not work. No error printed, no complain shown.
Environment:
- OS: centos 7.4
- CUDA: 10.0
- NVIDIA driver: 440.82
- GPU: GTX 1660
- python: 3.6
pytorch version: 1.6 and 1.4.
Any information are welcome, thanks in advance.
Edit:
According to Szymon's idea, running python3 foo.py, which with only import torch in it, and press ctrl-c, the prompt prints:
Traceback (most recent call last):
File "foo.py", line 1, in <module>
import torch
File "/usr/local/lib64/python3.6/site-packages/torch/__init__.py", line 48, in <module>
if platform.system() == 'Windows':
File "/usr/lib64/python3.6/platform.py", line 1068, in system
return uname().system
File "/usr/lib64/python3.6/platform.py", line 1034, in uname
processor = _syscmd_uname('-p', '')
File "/usr/lib64/python3.6/platform.py", line 788, in _syscmd_uname
f = os.popen('uname %s 2> %s' % (option, DEV_NULL))
File "/usr/lib64/python3.6/os.py", line 980, in popen
bufsize=buffering)
File "/usr/lib64/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib64/python3.6/subprocess.py", line 1318, in _execute_child
part = os.read(errpipe_read, 50000)
It seems that python hangs on running uname <option>, so I tried uname -a 2> /dev/null on command line, it returned immediately and nothing strange. Also created a file named bar.py with the content below:
import platform
print(platform.system())
and run it with python3, works well, printed 'Linux'. I don't think it is the reason, maybe just a coincidence?
I also tried more times, other situation looked like running import torch in python prompt, could not kill the process and nothing printed. Once it printed Soft lock up on CPU#4, I thought it was caused by the test process in the last disconnected session.