Cython subclassing "First base of" ... "is not an extension type" even though it is defined with cdef

user_13970481 0
Viewed 1215

I am working with Cython to optimize my Python code for a university project. To do that I want to transform python classes into extension types. I currently have issues compiling one extension type which should be a subclass of another extension type. This is the error I get:

src/core/ast/ast_classes/AstPreprocessor.pyx:9:27: First base of 'AstPreprocessor' is not an extension type

The definition of AstPreprocessor is as follows:

#Edit
from src.core.ast.ast_classes.AstBase import AstBase

cdef class AstPreprocessor(AstBase):
    cdef str function_name

    def __init__(self, function_ast, str function_name):
        super().__init__(function_ast)
        self.ast.index = self.ast.index.map(str)
        self.function_name = function_name
        self.symbol_list = super().get_symbol_list(self.function_name)

    #more method declarations

Here is part of the AstBase class including the method called in AstPreprocessor#__init__():

cdef class AstBase:
    cdef int length
    def __init__(self, df):
        self.ast = df
        self.length = int(df.shape[0])
        self.childrens = {}

    #more method declarations    

    cdef get_symbol_list(self, str function_name):
        symbol_list = []
        for i in self.ast.index:
            i = int(i)
            if self.token(i).startswith('SYMBOL') \
                    and self.text(i) != function_name:
                symbol_list.append(i)
        return symbol_list

Here is the cythonize command from my setup.py:

ext_modules=cythonize(["src/core/ast/ast_classes/*.pyx",
                       "src/core/ast/preprocessing/*.pyx"], 
                       language_level=3, annotate=True),

I have looked inside the docs but I have a hard time really understanding why this error occurs and how to fix it. This is the first time I am using Cython thus any help would be greatly appreciated.

Edit: I also tried using cimport but sadly the problem did not change.

cythonize subclassing cython python
1 Answers

You need to do 2 things. First create a .pxd file for AstBase called AstBase.pxd. These act sort of like C headers and are used to share Cython declarations between different modules. It should contain

cdef class AstBase:
    cdef int length
    # any other cdef attributes

    cdef get_symbol_list(self, str function_name)
    # but not the implementation of get_symbol_list

Your AstBase.pyx file looks substantially the same:

cdef class AstBase:
    def __init__(self, df):
        self.ast = df
        self.length = int(df.shape[0])
        self.childrens = {}

Note that I've removed length since it's declared in the pxd. Be aware that all attributes will need to be declared - currently ast and childrens aren't.

Then in AstPreprocessor.pyx you need to cimport rather than import AstBase

from AstBase cimport AstBase

# the rest stays the same

This ensures that Cython knows the details of the class (including the fact that it's a cdef class) at compile time. Typically if Cython doesn't know the details of an object it assumes that it's a regular Python object that'll be available at runtime, and this sometimes leads to confusing error messages.

user_46574120
Related