OSError: [WinError 1114] A dynamic link library (DLL) initialization routine failed, while loading torchtrt_runtime.dll in Python 3.10

Viewed 30

I was trying to load a compiled torch_tensorRT module on C++ and python.

In C++ I am loading it with:

const torch::Device device = torch::Device(torch::kCUDA, 0);
model = torch::jit::load(model_path, device);

In python I am loading it with:

torch.jit.load(model_path)

On C++ it worked fine on Debug mode, but on C++ Release mode and In Python it throwed following error:

RuntimeError: Unknown type name 'torch.torch.classes.tensorrt.Engine': File "code/torch/movinets/models.py", line 4 parameters = [] buffers = [] _torch___movinets_models_MoViNet_trt_engine : torch.torch.classes.tensorrt.Engine < -- HERE def forward(self_1: torch.movinets.models.MoViNet_trt, input_0: Tensor) -> Tensor:

However when I loaded torchtrt_runtime.dll on C++ release with following code, torch_tensorRT module got loaded perfectly:

HMODULE hLib = LoadLibrary(TEXT("torchtrt_runtime"));
    if (hLib == NULL) {
        std::cerr << "Library torchtrt_runtime.dll not found" << std::endl;
        exit(1);
    }

This makes it clear that running on windows need torchtrt_runtime.dll to be loaded. However when I'm trying to load torchtrt_runtime.dll in python with following code:

torch.ops.load_library("torchtrt_runtime.dll")

or

import ctypes
hllDll = ctypes.WinDLL("torchtrt_runtime.dll") or ctypes.CDLL("torchtrt_runtime.dll")

Library is not getting loaded and throwing following error:

hllDll = ctypes.WinDLL ("torchtrt_runtime.dll") File "C:\Users\NomanAnjum\AppData\Local\Programs\Python\Python310\lib\ctypes_init_.py", line 374, in init self._handle = _dlopen(self._name, mode) FileNotFoundError: Could not find module 'torchtrt_runtime.dll' (or one of its dependencies). Try using the full path with constructor syntax.

I then used dependancy checker for torchtrt_runtime.dll and added all dependant dlls to same folder and error has changed to:

File "video_play.py", line 198, in get_torch_tensorrt_converted_model hllDll = ctypes.WinDLL(r"torchtrt_runtime.dll") File "C:\Users\NomanAnjum\AppData\Local\Programs\Python\Python310\lib\ctypes_init_.py", line 374, in init self._handle = _dlopen(self._name, mode) OSError: [WinError 1114] A dynamic link library (DLL) initialization routine failed

I have this strong feeling that if I can load torchtrt_runtime.dll in python, it will run.

Please help me on this

0 Answers
Related