How to set the input to a LSTM network in C++

user_15295822 0
Viewed 80

I'm new to libtorch and I need to load a LSTM network in C++. Before that, I have already tested with the following Python script and it is working well:

actuator_net_file = "resources/actuator_nets/anydrive_v3_lstm.pt"
actuator_network = torch.jit.load(actuator_net_file)
actuator_network.eval()

num_envs = 1
num_actions = 1
sea_input = torch.zeros(num_envs*num_actions, 1, 2, requires_grad=False)
sea_hidden_state = torch.zeros(2, num_envs*num_actions, 8, requires_grad=False)
sea_cell_state = torch.zeros(2, num_envs*num_actions, 8, requires_grad=False)

torques, (sea_hidden_state[:], sea_cell_state[:]) = actuator_network(sea_input, (sea_hidden_state, sea_cell_state))

And the next step is to write a simple C++ program to test the forward evaluation of the network. But I don't know how to give arguments to the forward function. Here is what I got:

#include <torch/script.h> // One-stop header.
#include <torch/torch.h>

#include <iostream>
#include <memory>
#include <vector>

int main(int argc, const char* argv[]) {
  if (argc != 2) {
    std::cerr << "usage: example-app <path-to-exported-script-module>\n";
    return -1;
  }

  std::string actuator_net_file = "/home/fenglongsong/Desktop/example-app/anydrive_v3_lstm.pt";

  torch::jit::script::Module actuator_network;
  try {
    actuator_network = torch::jit::load(actuator_net_file);
    actuator_network.eval();
  }
  catch (const c10::Error& e) {
    std::cerr << "error loading the model\n";
    return -1;
  }

  std::cout << "load model ok\n";

  const int num_envs = 1;
  const int num_actions = 1;
  auto u0 = torch::zeros({num_envs*num_actions, 1, 2});
  auto h0 = torch::zeros({2, num_envs*num_actions, 8});
  auto c0 = torch::zeros({2, num_envs*num_actions, 8});

  std::vector<torch::jit::IValue> inputs;

  inputs.push_back(u0);
  std::vector<torch::jit::IValue> tuple;
  tuple.push_back(h0);
  tuple.push_back(c0);
  inputs.push_back(c10::ivalue::Tuple::create(tuple));

  std::cout << "before forward" << std::endl;
  actuator_network.forward(inputs).toTensor();
}

The compile passes successfully but when running the executable, the following error occurs:

fenglongsong@alvaro-rsl ~/Desktop/example-app/build $ ./example-app .
load model ok
before forward
terminate called after throwing an instance of 'c10::Error'
  what():  Expected Tensor but got Tuple
Exception raised from reportToTensorTypeError at ../aten/src/ATen/core/ivalue.cpp:908 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7f9153dc07ab in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xce (0x7f9153dbc15e in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libc10.so)
frame #2: c10::IValue::reportToTensorTypeError() const + 0x64 (0x7f913dd6d304 in /home/fenglongsong/Documents/ocs2_ws/src/libtorch/lib/libtorch_cpu.so)
frame #3: c10::IValue::toTensor() && + 0x4b (0x55d1ce3cd311 in ./example-app)
frame #4: main + 0x54e (0x55d1ce3ca0ec in ./example-app)
frame #5: __libc_start_main + 0xf3 (0x7f913c7c6083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #6: _start + 0x2e (0x55d1ce3c986e in ./example-app)

Aborted (core dumped)

My question is, what should be the equivalence in C++ of torques, (sea_hidden_state[:], sea_cell_state[:]) = actuator_network(sea_input, (sea_hidden_state, sea_cell_state)) ? Any suggessions will be much appreciated!

libtorch torch lstm c++
1 Answers

I have experience only directly with torch::Tensor, not with torch::jit::IValue, but generally, the signature is:

network.forward(const Tensor & input, torch::optional<std::tuple<Tensor,Tensor>>)

So in your example, you would call it like this:

actuator_network.forward(u0, std::make_tuple(h0, c0)).toTensor();

But it depends on the actual architecture, next time, include the actuator network architecture to your question to make it more clear.

user_88513060
Related