A error about use tensorrt to transfer the model of SRModel with dynamic shape inputs/output

Description

A error about use tensorrt to transfer the model of SRModel with dynamic shape inputs/output

Environment

TensorRT Version: 7.0.0.11 NVIDIA GPU: P100 NVIDIA Driver Version: 430.64 CUDA Version: 10.1 CUDNN Version*: 7.6.5 Operating System: Ubuntu 16.04 Python Version (if applicable): 3.7.0 PyTorch Version (if applicable): 1.7.0

Relevant Files

1. project files website: https://github.com/NVIDIA-AI-IOT/torch2trt
2. Model path: https://github.com/xinntao/BasicSR (srresnet_arch)

Steps To Reproduce

Before proposing this question, I have tried to do some test here:

1. Load the network in project, the shuffle_layer and interplote not work and I redefined a new class and refuse the issue.
2. I try to do three test to transfer this model by as fellow: A. I set the single input as (1,3,112,112) with output is (1,3,448,448), then I set dynamic as the office document (https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#runtime_dimensions). With the code as:

Build engine:

 input -> (1,3,112,112), inference_input -> (1,3,112,112), with output ->(1,3,448,448)
 with ConversionContext(network, torch2trt_kwargs=kwargs) as ctx:
        opt_shape_param = [(1,3,112,112),(1,3,224,224),(1,3,512,512)]
        config = builder.create_builder_config()
        profile = builder.create_optimization_profile()
        if isinstance(inputs, list):
            inputs = tuple(inputs)
        if not isinstance(inputs, tuple):
            inputs = (inputs,)
        ctx.add_inputs(inputs, input_names, opt_shape_param)
        outputsss = []
        profile.set_shape(network.get_input(0).name, opt_shape_param[0], opt_shape_param[1], opt_shape_param[2])
        for i, tensor in enumerate(inputs):
            outputs = module(tensor)
            if not isinstance(outputs, tuple) and not isinstance(outputs, list):
                outputsss += (outputs,)
        ctx.mark_outputs(outputsss, output_names)
        config.add_optimization_profile(profile)`

add input/ mark output:

 def add_inputs(self, torch_inputs, names=None):
    if names is None:
        names = default_input_names(len(torch_inputs))
    self.input_names = names
    for i, torch_input in enumerate(torch_inputs):
        if not hasattr(torch_input, "_trt"):
            trt_tensor = self.network.add_input(
                name=names[i],
                shape=tuple(torch_input.shape)[1:],
                dtype=torch_dtype_to_trt(torch_input.dtype),
            )
            trt_tensor.location = torch_device_to_trt(torch_input.device)
            torch_input._trt = trt_tensor
def mark_outputs(self, torch_outputs, names=None):
    if names is None:
        names = default_output_names(len(torch_outputs))
    self.output_names = names
    for i, torch_output in enumerate(torch_outputs):
        trt_tensor = torch_output._trt
        trt_tensor.name = names[i]
        trt_tensor.location = torch_device_to_trt(torch_output.device)
        trt_tensor.dtype = torch_dtype_to_trt(torch_output.dtype)
        self.network.mark_output(trt_tensor)

Inference:

    for i, input_name in enumerate(self.input_names):
        idx = self.engine.get_binding_index(input_name)
        self.context.set_binding_shape(idx, tuple(inputs[i].shape))
        bindings[idx] = inputs[i].contiguous().data_ptr()
    # create output tensors
    outputs = [None] * len(self.output_names)
    for i, output_name in enumerate(self.output_names):
        idx = self.engine.get_binding_index(output_name)
        dtype = torch_dtype_from_trt(self.engine.get_binding_dtype(idx))
        shape = tuple(self.context.get_binding_shape(idx))
        device = torch_device_from_trt(self.engine.get_location(idx))
        output = torch.empty(size=(1,3,448,448), dtype=dtype, device=device)
        outputs[i] = output
        bindings[idx] = output.data_ptr()`

Error: torch2trt.tests.torchvision.classification.msrresnet | float16 | [(1, 3, 112, 112)] | {'fp16_mode': True} | 3.03E-02 | 0.00 | 0.00E+00 | 0 | 0 | 0 | 0 |`

B. if I change the inference_input -> (1,3,224,224), with output ->(1,3,896,896) Error: Nothing is outputed and it seems the network is not work on the input`

C. Last, I change the input with input -> (1,3,112,112), (1,3,224,224) with output (1,3,448,448), (1,3,896,896) through operate fellow code two times profile.set_shape(network.get_input(0).name, opt_shape_param[0], opt_shape_param[1], opt_shape_param[2]) Meantime I change the bindings:[None, None, None, None] or bindings: [None, None] and push the input and output number into corresponding position. **Error:** Nothing is outputed and it seems the network is not work on the input

Others: I try to set the add_input for network as fellow: network_definition.add_input("foo", trt.float32,(3, -1, -1)) A new issue is occuring : ERROR: Parameter check failed at: ../builder/Network.cpp::addInput::957, condition: isValidDims(dims, hasImplicitBatchDimension())

How I work it and whether my thinking is wrong ? Please help me !