Does numpy.i send actual pointers to C++, or make copies of memory?

user_11501167 0
Viewed 84

I am learning about C++, swig, and numpy for a lab I am working in. It has been specified that I must use swig (no scypy), I must use numpy arrays, and I may not "bring C code into the python world" such as by using %import "std_vector" in my interface file and just having the python user create one to send in. That being said, I am trying to get a 1d numpy array (if they need more dimensions I will just flatten it) to be passed into my C code by pointer exclusively - my boss doesn't want to have to take the time to copy everything because efficiency is very important. I believe we use c++ 14, python 2.7, and the latest version of swig, and I am using numpy.i as well.

I will provide the code below that I am currently using (just trying to get a minimum viable going here) but I am pretty new, and while it does work, I am not sure that is actually passing a pointer and not copying anything like I would like. Could someone please either confirm that it is, or show me how to make it do so? Thanks x10^99

//The C++ file I am wrapping: 
#ifndef _np_array_to_array_h
#define _np_array_to_array_h

using namespace std;
double getMid(double* myArray, int size){
    int half = size / 2;
    return myArray[half];
}
#endif

///\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\//\/\

//The interface file: 
%module np_array_to_array

%{
#define SWIG_FILE_WITH_INIT
#include "np_array_to_array.h"
#include <numpy/arrayobject.h>
%}
%include "numpy.i"
%init %{
import_array();
%}
%apply (double* IN_ARRAY1, int DIM1){(double* myArray, int size)};
%include "np_array_to_array.h"

//\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/

//How I compile on terminal: 
swig -python -c++ np_array_to_array.i
g++ -fpic -c np_array_to_array_wrap.cxx -I/usr/include/python2.7 
-I/home/sean/Desktop/SerangLab/Swig/numpy/numpy/core/include/ -I/home/sean/.local/lib/python2.7/site-packages/numpy/core/include/numpy/
 g++ -shared np_array_to_array_wrap.o -o _np_array_to_array.so

This will compile and run, and create a successfully working python module that I import with (when in the same directory) "from np_array_to_array import *" and I can successfully run the getMid method, passing in a numpyArray and getting a double back out. As stated above, I am simply not sure if this is actually pass by pointer (not making any copies) or not, as I have not found anything that says one way or the other. Could someone please tell me, and if it is not, explain how one would do this? I believe it should be possible as I think numpy array uses c types and stores memory contiguously like c does.

numpy-ndarray swig numpy python c++
1 Answers

You can investigate this fairly easily in the code that SWIG generates. There are two parts to this - a .py file and a .cxx file. In each of these there's some code generate for your getMid() function. The Python code just directly passes everything into the C++ code, which on my system ended up looking like this:

SWIGINTERN PyObject *_wrap_getMid(PyObject *SWIGUNUSEDPARM(self), PyObject *args) {
  PyObject *resultobj = 0;
  double *arg1 = (double *) 0 ;
  int arg2 ;
  PyArrayObject *array1 = NULL ;
  int is_new_object1 = 0 ;
  PyObject * obj0 = 0 ;
  double result;

  if (!PyArg_ParseTuple(args,(char *)"O:getMid",&obj0)) SWIG_fail;
  {
    npy_intp size[1] = {
      -1 
    };
    array1 = obj_to_array_contiguous_force_conversion(obj0, NPY_DOUBLE,
      &is_new_object1);
    if (!array1 || !require_dimensions(array1, 1) ||
      !require_size(array1, size, 1)) SWIG_fail;
    arg1 = (double*) array_data(array1);
    arg2 = (int) array_size(array1,0);
  }
  result = (double)getMid(arg1,arg2);
  resultobj = SWIG_From_double(static_cast< double >(result));
  {
    if (is_new_object1 && array1)
    {
      Py_DECREF(array1); 
    }
  }
  return resultobj;
fail:
...

It won't change much between SWIG and Python versions, although some of the SWIG options will change it a little bit.

The important point from the perspective of your question though seems to be the call to obj_to_array_contiguous_force_conversion. It has an argument that's used as an output parameter to indicate if a new object was allocated. If that ends up being set to true then after the call an object ends up getting released also.

From that alone it's pretty safe to conclude that the answer to your question is that it depends what input you pass to the function. If it already satisfies the constraints (i.e. it's contiguous) then you won't end up making a copy. Otherwise it will, since your C++ function requires a contiguous region.

It should also be a safe bet that if you use any of the numpy double types then you'll end up meeting this requirement and not making a copy, but for other data types that's less likely unless you've gone to a bit of effort.

user_1681750
Related