Cython inplace assignment of values for a libcpp.list

user_1459254 0
Viewed 364

Is there a way to make a dereference before assignment in Cython, in order to do e.g. inplace assignment of values in a C++ list?

Cython documentation (here) says that you should use dereference(foo) for producing C++ code like *(foo), but this syntax cannot be used for assignment (e.g. *(foo) = *(foo) + 1.

Below I show an example of assignment (in the inplace_normalize function) that I couldn't implement using the Cython dereference function.

from libcpp.list cimport list as cpplist
from cython.operator import dereference as deref, preincrement as inc

cpdef cpplist[double] normalize(cpplist[double] l):
    cdef double norm
    cdef cpplist[double] l2

    for v in l:
        norm += v

    for v in l:
        l2.push_back(v/norm)

    return l2

# inplace version of normalize  
cpdef void inplace_normalize(cpplist[double] l):
    cdef double norm
    for v in l:
        norm += v

    cdef cpplist[double].iterator it = l.begin()
    while it != l.end():
        # deref(it) = deref(it)/norm  #== *it = *it / norm
        print(deref(it)/norm)
        inc(it)

If I try to use deref(it) = deref(it)/norm, I get the error Cannot assign to or delete this. What is the solution for the inplace assigment of values in a libcpp.list using Cython?

cython list c++
2 Answers

With a pointer you can use pt[0] instead of (*pt) because pointers and arrays are closely related in C. However the same trick doesn't quite work with C++ iterators. Therefore what we need to do is to get the address of the item. A rough (untested) outline is:

cdef void inplace_normalize(cpplist[double]& l):
    cdef double norm
    cdef double* address
    for v in l:
        norm += v

    cdef cpplist[double].iterator it = l.begin()
    while it != l.end():
        address = &deref(it)
        address[0] = address[0]/norm  #== *it = *it / norm
        print(deref(it)/norm)
        inc(it)

A few side points:

  • C++ list is very rarely what you want - you probably want C++ vector instead, which is a closer match to the Python list and usually more efficient.

  • I've changed the function from cpdef to cdef. If you tried calling it from Python it would first copy the Python iterable into a C++ list, then it would normalise the contents of that list, but the changes you made would never be propagated back into the iterable. There's no way it makes sense for this to be callable from Python (since you can't have cpplist objects directly in Python) so it's best to block this.

  • I've changed it from pass-by-value to pass-by-reference. It doesn't make sense to be modifying a local copy of a list.

user_46574120

@DavidW's solution is what you have asked for, however I would suggest a slightly different approach when coding C++ in Cython.

In Cython, C++ always felt like a second-class citizen (just look at this dereference-mess or missing functionality in libcpp-wrappers!) and since "verbatim-C-code" (i.e. since Cython-0.28) there are no good reasons for this torture - just inline the c++-code verbatim and have all of c++-functionality at your disposal:

# make sure c++11 is enabled
%%cython --cplus

from libcpp.list cimport list as cpplist

cdef extern from *:
    """
    #include <list>
    #include <algorithm>
    #include <numeric>
    void normalize_inplace_cpp(std::list<double>& lst){
        double norm = std::accumulate(lst.cbegin(), lst.cend(), 0.0)/lst.size();
        std::for_each(lst.begin(), lst.end(), [norm](double &val){
           val/=norm;
        });
    }
    """
    void normalize_inplace_cpp(cpplist[double]& lst)

# for testing
def normalized_list(cpplist[double] lst):
    normalize_inplace_cpp(lst)
    return lst

And now:

>>> normalized_list([1,2,3])
# [0.5, 1.0, 1.5]
user_57694630
Related