std::vector, how does it store values with libtorch tensors?

user_18181751 0
Viewed 340

When I was collecting trainable parameters as vector<torch::tensor>, I've realized that it is type cast to torch::autograd::VariableList.

With this structure, how does the vector access its element? Does it store the value's memory space even without explicitly having to call them by pointer or reference?

So I've tested with some simple codes like this. With regular int data type:

int a = 10;
std::vector<int> b;
b.push_back(a);
b[0] += 10;
cout << b << endl;
cout << a << endl;

As expected, this produces 20 for b (only one element), and 10 for a (original int data)

However, for the torch::Tensor with the same style of codes:

torch::Tensor t = torch::ones({ 1 });
std::vector<torch::Tensor> tv;
tv.push_back(t);
tv[0] += 10;
cout << t << endl;
cout << tv << endl;

Just like the int vectors, I thought tv will produce 11 (one element vector), and v is just 1 (shape 1) However, the results for both tv and v are updated to 11. Although the operation is done on the vector, the original tensor value is also updated. Why does this happen? My guess is torch::autograd::Variable list stores its element by memory address...?

Also, when you do,

torch::Tensor t = torch::ones({ 1 });
std::vector<torch::Tensor> tv;
tv.push_back(t);
tv[0] = tv[0] + 10;
cout << t << endl;
cout << tv << endl;

Only tv value is updated to 11 and original tensor t is the same 1.

I mean this makes collecting trainable parameters and passing them to the optimizer much easier, but I am really not sure about how this happens. Could you please kindly explain to me why these cases are all different and how the vector stores the elements in each case?

Thank you very much for your help in advance!

libtorch pytorch std vector c++
1 Answers

This is not strange behaviour of std::vector, it is strange behaviour of torch::Tensor. The following should also exhibit it.

int a = 10;
int b = a;
b += 10;
std::cout << b << std::endl;
std::cout << a << std::endl;

torch::Tensor c = torch::ones({ 1 });
torch::Tensor d = c;
d += 10;
std::cout << d << std::endl;
std::cout << c << std::endl;

torch::Tensor e = torch::ones({ 1 });
torch::Tensor f = e;
f = f + 10;
std::cout << f << std::endl;
std::cout << e << std::endl;

A std::vector<T> allocates some space, and constructs T instances in that space. The particular constructor used depends on how you insert. In your case push_back uses the copy constructor of the type (it would use the move constructor if given an rvalue).

user_26108100
Related