I want to create custom Splitter and DecisionTreeRegressor classes to be used with RandomForestRegressor in sklearn. However, the Splitter class is written in Cython. The sklearn GitHub repo has two splitter files, one pxd and one pyd: https://github.com/scikit-learn/scikit-learn/tree/fd237278e895b42abe8d8d09105cbb82dc2cbba7/sklearn/tree
What I've read about Cython indicates that there is a manual compilation step required to use Cython code, but I would think (or hope) that I could just manually edit the _splitter.pxd and _splitter.pyd files since that's how they are laid out in the GitHub repo and I'm obviously not compiling anything when I use sklearn. However, I get a couple of errors when trying to do this on macOS.
Using the Spyder IDE, I get an invalid syntax error from this line in _splitter.pxd when I try to make a copy of that file and import my copied version rather than the sklearn version:
cimport numpy as np
Again, this line is identical to the cimport statement in _splitter.pxd in sklearn: https://github.com/scikit-learn/scikit-learn/blob/fd237278e895b42abe8d8d09105cbb82dc2cbba7/sklearn/tree/_splitter.pxd
Also, when trying to import the splitter file by mirroring the import in sklearn, I get a No module named... error:
import sklearn_custom.tree._splitter
I double checked that this is not just an issue with Spyder by running it on the command line and still got the error.
So I'm wondering what python/sklearn are doing behind the scenes that allow the Cython code to run as is and without compilation or any of the Cython-specific imports that are found in the Cython instructions (https://cython.readthedocs.io/en/latest/src/tutorial/cython_tutorial.html). Is compiling unavoidable if I want to write custom Cython to be used with sklearn? If so, what exactly do I do with the existing pyx and pxd files (mainly the latter since there's no mention of pxd in the Cython instructions and the pxd file is not imported into the pyx file) that already exist for sklearn?