I'm working on a Cython project involving multiple extension modules that need to communicate their classes between each other.
I'll add example at the end, but my boiled-down issue is this: say I have two c++ classes, one called Meal and one called Jimmy. Both classes are wrapped as cdef class Py[Meal/Jimmy] extension types, living in files named [Meal/Jimmy]Wrapper.[pxd/pyx].
A PyMeal is created by a Manager, loaded up with Food (same deal, wrapped in PyFood, but not causing problems), and then served to our friend PyJimmy. PyJimmy then collects the Meal from the PyMeal, stores it in his wrapped Jimmy, and the process begins anew.
Concerning the structure of this example, the wrappers for Jimmy & Meal are both two modules deep (example.consumers.JimmyWrapper and example.items.MealWrapper) and a manager .pyx is one deep (example.MealtimeManager).
The issue lies in that last step: when PyJimmy is served, he is then to take the Meal pointer from the PyMeal and store it in his own Jimmy. To do this, I presume PyJimmy should have some method to accept a PyMeal, which is then passed to a cdef function to grab the Meal pointer. However, cimporting example.items.MealWrapper in Jimmy's .pxd only exposes the base Meal type. Not surprising, as PyMeal isn't mentioned in the pxd.
I've tried defining cdef class PyMeal in the MealWrapper.pxd, but then the same statement in MealWrapper.pyx fails, citing C class 'PyMeal' is declared but not defined.
However, if I skirt around that error with some not-actually-a-fix fix, putting any cdef class [Anything] after the extern cppclass bit in MealWrapper.pxd breaks everything. When the Manager goes to import the wrapper Cython extensions (import MealWrapper & import JimmyWrapper), the cimport in [Jimmy/Meal]Wrapper.pyx throws an error on execution, not compile: ModuleNotFoundError: No module named 'example.items.MealWrapper'.
Minimal Example:
Environment: Windows 10, Anaconda installation of Python 3.8.8, Cython 0.29.22
setup.py
test.py # the bit that creates the MealtimeManager
cpp/* # Jimmy/Meal C++ headers/implementations
example/__init__.py # empty
example/MealtimeManager.pyx
example/items/__init__.pxd # empty
example/items/__init__.py # empty
example/items/MealWrapper.pxd
example/items/MealWrapper.pyx
example/consumers/__init__.pxd # empty
example/consumers/__init__.py # empty
example/consumers/JimmyWrapper.pxd
example/consumers/JimmyWrapper.pyx
# setup.py
# omitting imports
ext_modules = [
Extension("JimmyWrapper",
["./cpp/Jimmy.cpp", "./example/consumers/JimmyWrapper.pyx"],
language='c++'),
Extension("MealWrapper",
["./cpp/Meal.cpp", "./example/items/MealWrapper.pyx"],
language='c++'),
Extension("MealtimeManager",
["./example/items/MealtimeManager.pyx"]),
]
setup(cmdclass={'build_ext': build_ext},
ext_modules=ext_modules, build_dir="build")
# MealtimeManager.pyx
#!python
#cython: language_level=3
from JimmyWrapper import PyJimmy
from MealWrapper import PyMeal
def run(): # this function is just for example of what I aim to do
jim = PyJimmy()
steak_dinner = PyMeal()
jim.eat_meal(steak_dinner)
# MealWrapper.pxd
cdef extern from "../../cpp/Meal.h":
cdef cppclass Meal:
Meal() except +
# cdef class ThisIsJustForExample: pass # uncommenting this will cause the error included at the bottom
# MealWrapper.pyx
#!python
#cython: language_level=3
from example.items.MealWrapper cimport Meal
cdef class PyMeal:
cdef Meal * thisptr
def __cinit__(self):
self.thisptr = new Meal()
def __dealloc__(self):
del self.thisptr
# For sake of space, I'm omitting the bits that deal with food here.
# However, know that the cppclass Food would be declared in MealWrapper.pxd
# and PyFood implemented in this file, which does allow for a function
# in PyMeal that looks for a PyFood parameter.
# JimmyWrapper.pxd
from example.items.MealWrapper cimport Meal # this bit works fine
cdef extern from "../../cpp/Jimmy.h":
cdef cppclass Jimmy:
Meal * last_meal
Jimmy() except +
void eat_meal(Meal * m)
# JimmyWrapper.pyx
#!python
#cython: language_level=3
from example.consumers.JimmyWrapper cimport Jimmy
###### here's where I want to import PyMeal
#from example.items.MealWrapper import Meal # this imports the base class, which I can't find a way to use
#from example.items.MealWrapper import PyMeal # is what I'm theoretically looking to do
######
cdef class PyJimmy:
cdef Jimmy thisptr
def __cinit__(self):
self.thisptr = Jimmy()
## These are the functions that I imagine I'll require for this,
## naturally both can't find PyMeal.
cdef void _eat_meal(self, PyMeal m):
self.thisptr.eat_meal( m.mealptr ) # calls base class eat
cpdef eat_meal(self, PyMeal m):
self._eat_meal( m ) # calls cdef PyJimmy eat
# example error after trying to put PyMeal cdef in MealWrapper.pxd
Traceback (most recent call last):
File ".\test.py", line 29, in main
import MealtimeManager
File "example\MealtimeManager.pyx", line 4, in init MealtimeManager
from JimmyWrapper import PyComputer
File "py\consumers\JimmyWrapper.pyx", line 1, in init JimmyWrapper
#!python
ModuleNotFoundError: No module named 'example.items.MealWrapper'
The solutions I've tried were gathered from other questions about extension types and importing and what an init.pxd actually does, but those didn't seem too similar or fix it. I can't tell if my issue is with how I've structured everything, I'm trying to import the wrong thing, there's some Cython bug, or I'm just plain lookin' at it wrong. If I'm forgetting to include anything, do let me know and I'll quickly slap an edit on this bad boy.
Thanks for any and all help!