Is it possible to override a cdef method with def? Override method won't execute

user_11677720 0
Viewed 610

I am implementing a class MyBinaryTissueClassifier that has superclass TissueClassifier (which I do not have the ability to change). I am attempting to override one of the methods in TissueClassifier in MyBinaryTissueClassifier but my implementation is not getting called. The superclass function is a cdef function and I am using Python and attempting to override with a def function. Is this why the code is not overriding?

The TissueClassifier class is as follows:

cdef class TissueClassifier:
    *code elided*
    cdef TissueClass check_point_c(self, double* point)
        pass

MyBinaryTissueClassifier class should override it and it looks like this:

class MyBinaryTissueClassifier(TissueClassifier):
    *code elided*
    def check_point_c(self, point):
        *method body here*

I was expecting the body of the MyBinaryTissueClassifier to execute but it is never called, nor is the original instance of BinaryTissueClassifier. It seems that the TissueClassifier check_point_c() method stops after it calls pass.

cython python
1 Answers

cdef function can be only overriden by a cdef (or cpdef) function in the derived class. This is also the reason, cython emits warning:

warning: xxxxx.pyx:yy:z: Overriding cdef method with def method.

for the example above.

The usual def-function is very flexible: Not only multiple inheritances must be taken into account, but for example also monkey-patching. But that also means there is not much place for optimization: there is not much more Cython can do than using python's machinery, i.e. PyObject_GetAttr/PyObject_GenericGetAttr + PyMethod_GET_SELF + PyMethod_GET_FUNCTION+ call of the function.

This flexibility leads to large overhead, so cdef-functions avoid it by using a different strategy, which is similar to C++-virtual functions:

  • Every cdef-class has a virtual table, where the pointer to cdef-functions are stored, every one in its own slot.
  • Every object of this cdef-class has a pointer to this virtual table thus the calls to cdef-functions can be resolved at the run time and the resolution costs only an indirection.
  • A subclass can override a cdef-function by putting another function-pointer into the correspoding slot in the virtual table.

A typicall call of the cdef-function looks as follows:

%%cython

cdef class A:
    cdef doit(self):
        print(42)
    def call_doit(self):
        self.doit()

The self.doit() leads to the following c-code:

((struct __pyx_vtabstruct_XXX_A*)__pyx_v_self->__pyx_vtab)->doit(__pyx_v_self, 0);

The v-table of the object __pyx_v_self is interpreted as v-table of the cdef-class A (even if self is not an instance of A but of a derived class, this operation works correctly) and slot doit is executed.

When a class, let's say B derives from A it can override the slot doit, so B's version of the doit were called. The slot doit is overriden by providing a corresponding definition of cdef-function.

As one can see, there are different machineries at play when a def function is called compared to a cdef-function.

Thus cdef functions are overriden only by other cdef-functions (and not def-functions). Being more precise, one can also use a cpdef-method for overwriting - but in your example it will not work because cpdef functions cannot have double * arguments, as they cannot be converted automatically by Cython to Python-objects,and the signatures of the overriding and overrided functions must match.

user_57694630
Related