Unable to import .so file in python

Viewed 1562

Note that this is all being run in Google Colab.

The .so extension module was written in C++/Cuda and built using Ninja.

The issue I'm having is understanding why I am unable to import this .so extension module.

As I said, I am using Google Colab. What is incredibly confusing is that when I cd into the directory containing the .so extension module _ext, I am able to import it perfectly fine, and all intended functionality works. However, when I try to import it from a .py file through a series of imports,

I.e.

File x imports y which imports z which imports said .so extension module.

it says that the module cannot be found. I've tried adding the file to sys.path, but nothing works.

Below is the file folder structure, and the colab code/error output.

File Folder Structure

├── CenterNet
│   ├── data
│   ├── exp
│   ├── experiments
│   ├── images
│   ├── LICENSE
│   ├── models
│   ├── NOTICE
│   ├── readme
│   ├── README.md
│   ├── requirements.txt
│   └── src
│       ├── demo.py
│       ├── _init_paths.py
│       ├── lib
│       │   ├── datasets
│       │   ├── detectors
│       │   ├── external
│       │   ├── logger.py
│       │   ├── models
│       │   │   ├── data_parallel.py
│       │   │   ├── decode.py
│       │   │   ├── losses.py
│       │   │   ├── model.py
│       │   │   ├── networks
│       │   │   │   ├── DCNv2
│       │   │   │   │   ├── build
│       │   │   │   │   │   ├── lib.linux-x86_64-3.6
│       │   │   │   │   │   │   └── _ext.cpython-36m-x86_64-linux-gnu.so
│       │   │   │   │   │   └── temp.linux-x86_64-3.6
│       │   │   │   │   ├── DCNv2.egg-info
│       │   │   │   │   ├── dcn_v2_onnx.py
│       │   │   │   │   ├── dcn_v2.py
│       │   │   │   │   ├── _ext.cpython-36m-x86_64-linux-gnu.so # <-- The file I want to import.
│       │   │   │   │   ├── __init__.py
│       │   │   │   │   ├── LICENSE
│       │   │   │   │   ├── make.sh
│       │   │   │   │   ├── README.md
│       │   │   │   │   ├── setup.py
│       │   │   │   │   ├── src
│       │   │   │   │   │   ├── cpu
│       │   │   │   │   │   ├── cuda
│       │   │   │   │   │   ├── dcn_v2.h
│       │   │   │   │   │   └── vision.cpp
│       │   │   │   │   ├── testcpu.py
│       │   │   │   │   └── testcuda.py
│       │   │   │   ├── dlav0.py
│       │   │   │   ├── large_hourglass.py
│       │   │   │   ├── msra_resnet.py
│       │   │   │   ├── pose_dla_dcn.py
│       │   │   │   ├── __pycache__
│       │   │   │   └── resnet_dcn.py
│       │   │   ├── __pycache__
│       │   │   │   ├── decode.cpython-36.pyc
│       │   │   │   ├── model.cpython-36.pyc
│       │   │   │   └── utils.cpython-36.pyc
│       │   │   ├── scatter_gather.py
│       │   │   └── utils.py
│       │   ├── opts.py
│       │   ├── __pycache__
│       │   ├── trains
│       │   └── utils
│       ├── main.py
│       ├── test.py
│       └── tools

Colab Code

    %cd $root_dir
    %cd CenterNet/src/lib/models/networks/DCNv2
    
    !ls
    
    import _ext 
    import sys
    
    print('I exist at - ', _ext.__file__, ' !')
    sys.path.insert(0, _ext.__file__)
    
    %cd $root_dir/CenterNet/src
    
    !python demo.py ctdet --demo /content/CenterNet/images/19064748793_bb942deea1_k.jpg --load_model ../models/ddd_3dop.pth

Output

/content
/content/CenterNet/src/lib/models/networks/DCNv2

build       _ext.cpython-36m-x86_64-linux-gnu.so  __pycache__  testcpu.py
DCNv2.egg-info  __init__.py               README.md    testcuda.py
dcn_v2_onnx.py  LICENSE                   setup.py
dcn_v2.py   make.sh                   src

I exist at -  /content/CenterNet/src/lib/models/networks/DCNv2/_ext.cpython-36m-x86_64-linux-gnu.so !
/content/CenterNet/src

['/content/CenterNet/src/lib/models/networks/DCNv2/_ext.cpython-36m-x86_64-linux-gnu.so', '/content/CenterNet/src/lib', '/content/CenterNet/src', '/env/python', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/root/.local/lib/python3.6/site-packages', '/root/.local/lib/python3.6/site-packages/pycocotools-2.0-py3.6-linux-x86_64.egg', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages']

Traceback (most recent call last):
  File "demo.py", line 16, in <module>
    from detectors.detector_factory import detector_factory
  File "/content/CenterNet/src/lib/detectors/detector_factory.py", line 5, in <module>
    from .exdet import ExdetDetector
  File "/content/CenterNet/src/lib/detectors/exdet.py", line 21, in <module>
    from .base_detector import BaseDetector
  File "/content/CenterNet/src/lib/detectors/base_detector.py", line 11, in <module>
    from models.model import create_model, load_model
  File "/content/CenterNet/src/lib/models/model.py", line 12, in <module>
    from .networks.pose_dla_dcn import get_pose_net as get_dla_dcn
  File "/content/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 16, in <module>
    from .DCNv2.dcn_v2 import DCN
  File "/content/CenterNet/src/lib/models/networks/DCNv2/dcn_v2.py", line 17, in <module>
    import _ext as _backend
ModuleNotFoundError: No module named '_ext'

I have no idea why I am able to import the _ext module when I cd into the directory it is contained but unable to import it otherwise; even when added to sys.path! Any help would be greatly appreciated.

0 Answers
Related