how to transfer ownership from a c++ std::vector<char> to a char *pointer

user_12971681 0
Viewed 1034

I have met a situation that, i got a std::vector<char> data output from a third-party lib, it is a very big length data, and then I need to convert it a pybind11::array object, I don't want to allocate memory and do memcpy ,that's not efficient at all.

Now I know I can got the std::vector<char> buffer address, but I do not know how to release the vector's ownership so the buffer would not release when the vector object is destructed, I wonder if there's a way to achieve this,.

I have wrote a test code below to test, but it failed

#include<vector>
#include<iostream>
int *got_vec(int len){// the len in actually scene is decided by the thirdparty lib

  std::vector<int> vec;
  for(int i =0;i<len;i++){
    vec.push_back(i);
  }
  int *p_vec = &vec[0];
  std::move(vec);
  return p_vec;
}

int main(int argc,const char **argv){
  int len=atoi(argv[1]);

  std::cout<<"will go allocate memory size:"<<len<<", before allocation"<<std::endl; 

  int *p_vec = got_vec(len);
  std::cout<<"after allocation, go to print value"<<std::endl; 
  for(int i = 0; i < len;i++)
   std::cout<<p_vec[i]<<",";
  std::cout<<std::endl;

  delete p_vec;
  std::cout<<"deleted"<<std::endl;
}

The program crashed at std::cout<<p_vec[i]<<",";

pybind11 c++
4 Answers
user_128678880

std::vector doesn't provide way to transfer its buffer.

You have to do some copy (or not use std::vector for original buffer).

user_26845390

You can of course, because this is what the std::vector class does for move operation. But you cannot in a portable way (*). You must find in the internals of the std::vector class of your implementation, how the buffer is stored (what attribute is used). Ideally you should search how what the move constructor does with the source vector and do the same. It is likely to just set the internal pointer to NULL, but you should control that.

Simply as you would be using unspecified internals, it would only be guaranteed to work on that version of that compiler.

On a philosophical point of view, we as the programmers are expected to use the classes of the standard library as they are. There are few extension points, and few classes allow to be derived. It is just the opposite of Java which provides abstract classes to help the programmer to build its own custom classes.

user_35452730

You can't directly 'steal' memory from an std::vector, but maybe you don't need to steal it in the 1st place.

I'm not familiar with pybind11::array, but since you want the data pointer from std::vector, I'm guessing you can construct it from data that was allocated somewhere else.

Maybe all you need is a wrapper class, that would keep your data inside a std::vector, while providing a view into it through pybind11::array.

class Wrapper {
public:
  Wrapper(std::vector<char>);
  pybind11::array asArray();

private:
  std::vector<char> m_data
}

You can then use std::move to efficiently transfer data between std::vectors, without copying it around.

user_83492520
Related