how to pass float* array to C method from python script whose memory allocation takes place in C layer

user_7074480 0
Viewed 463

I am trying to call C methods from Python script, C method calls inturn the C++ method. I am allocating array inside the getResults() method using malloc(). Now the issue is how to pass the arguments to float* oresults in python script whose memory allocation takes place inside the C layer. This is io.c

int getResults(char* iFilename, char* iStagename, int iStateidCnt, 
    int* Stateids, int iEntityIdCount, int* iEntityids, char* iEntityType,
    char* iVariablegroup, char* ivariable, int *oRescount,
    float* oResults)
{
    int Status, i;
        EString etype(iEntityType), stagename(iStagename);
    EString vargroup(iVariablegroup);
    std::vector<ERF_INT> entity_ids;
    std::vector<ERF_INT> stateids;
    std::vector<ERF_FLOAT> results;
    _CopyIntArrayIntoVector(iStateidCnt, Stateids, stateids);
    _CopyIntArrayIntoVector(iEntityIdCount, iEntityids, entity_ids);
    CreateIoInstance(iFilename, iStagename);
    ioData pIodata = CreateIoDataInstance();
    if (iEntityIdCount <= 0)
        pIodata.setWholeSection(true);
    else
    {
        pIodata.setWholeSection(false);
        pIodata.setEntityList(entity_ids);
    }
        
    pIodata.setStateList(stateids);
    pIodata.setType(etype);
    pIodata.setVariableGroup(iVariablegroup);
    pIodata.setVariable(ivariable);
        //This is C++ method
    Status = pIo->get_results(pIodata, results);
    *oRescount = results.size();
        //allocation for oresults whose size > 2
    oResults = (float*)malloc(results.size() * sizeof(float));
    _CopyVectorIntoDoubleArray(results, oResults);
    return Status;
}

test.py

from ctypes import *
import os, sys
dll = CDLL('D:\\erf_utils_python\\erf_utils_io.dll')
dll.getresults.argtypes = (c_char_p,c_char_p,c_int,POINTER(c_int),c_int,POINTER(c_int),c_char_p,
                                  c_char_p,c_char_p,POINTER(c_int),POINTER(c_float))
dll.getresults.restype = c_int


def make_array(ctype,arr):
    return len(arr),(ctype * len(arr))(*arr)

def getresults(filename,stagename,sids,eids,entitytype,groups,variables):
    if(len(sids)>0):
       stateidcount,stateids = make_array(c_int,sids)
    if(len(eids)>0):
       entityidcount,entityid = make_array(c_int,eids)
    oresultlen = c_int()
    float_values = POINTER(c_float)
    err = dll.getresults(filename,stagename,stateidcount,stateids,entityidcount,entityid,
                                entitytype,groups,variables,byref(oresultlen), byref(float_values))
    return err,oresultlen.value, float_values

filename = b'D:\\inputfile.h5'
stagename = b"post"
stateids = [2]
stateidcount = 1
entityidcount = 1
entityid = [1]
entitytype = b"test"
variablecount = 1
variablegroup = b"testdata"
variable = b"next"

err,oreslen,ores = getresults(filename,stagename,stateids,entityid,entitytype,variablegroup,variable)

TypeError: byref() argument must be a ctypes instance, not '_ctypes.PyCPointerType' this is the error I get when I run the script. I am little confused on how to send argument for float *oresults in script.

ctypes c arrays python c++
2 Answers

In the C++ code, the signature int getResults(..., float* oResults) isn't able to communicate the allocated pointer back to the caller. The line

oResults = (float*)malloc(results.size() * sizeof(float));

sets the oResults pointer locally within getResults, without affecting the caller. In order to output a pointer, you must either return it or use a pointer-to-pointer argument: int getResults(..., float** oResults).

In the Python code, I'm not familiar with ctypes but it looks like float_values = POINTER(c_float) is a problem. POINTER(c_float) creates a Python type for a pointer to float. You'd want POINTER(c_float)() to make an instance of such a pointer (which is initially null).

ctypes documentation on pointers: https://docs.python.org/3/library/ctypes.html#pointers

user_132239860

The float* oResults parameter is passed by value, so it is impossible to return an allocated pointer in that parameter. Instead, use a float** oResults.

Also, float_values = POINTER(c_float) is a type, not an instance of a type. So byref(float_values) is equivalent to the invalid C &(float*). Instead, you want an instance of a pointer POINTER(c_float)() (note parentheses) and pass that by reference, similar to C float *p; func(&p). This will pass the pointer by address to the C function, which can then modify it as an output parameter.

Here's a simplified example focusing on just the int *oRescount and float** oResults parameters. Also needed is a function to free the allocation:

test.cpp

#include <vector>
#define API __declspec(dllexport)

extern "C" {
    API int getResults(size_t *oRescount, float** oResults) {
        std::vector<float> results {1.25,2.5,3.75,5.0}; // Simulated results
        *oRescount = results.size(); // Return size of results
        auto tmp = new float[results.size()]; // allocate
        for(size_t i = 0; i < results.size(); ++i) // copy vector to allocation
            tmp[i] = results[i];
        *oResults = tmp; // return allocation
        return 0;
    }

    API void freeResults(float* oResults) {
        delete [] oResults;
    }
}

test.py

from ctypes import *
dll = CDLL('./test')
dll.getResults.argtypes = POINTER(c_size_t),POINTER(POINTER(c_float))
dll.getResults.restype = c_int

def getresults():
    oRescount = c_size_t()         # instance to hold the returned size
    oResults = POINTER(c_float)()  # instance of a float* to hold the returned allocation.
    err = dll.getResults(byref(oRescount), byref(oResults))

    # oResults is a float* and it is possible to index past the end.
    # Make a copy into a Python list slicing to the correct size,
    # then free it so there is no memory leak.
    results = oResults[:oRescount.value]
    dll.freeResults(oResults)

    return err,results

err,ores = getresults()
print(err,ores)

Output:

0 [1.25, 2.5, 3.75, 5.0]
user_2356980
Related