How to specify the target directory for a AOT compiled numba extension?

user_9122857 0
Viewed 421

I am trying to create a few AOT compiled python code for a project using numba.

I would like to have all my source code in a particular directory and then use these to build the compiled code through the conventional setup.py in a different directory. For example, let's assume my project is structured as follows

- numba_project/
    - numba_project/
        - __init__.py
        - src/
            - numba_source.py
        - lib/
    - setup.py

The numba_source.py looks something like

from numba.pycc import CC

cc = CC('my_module')

@cc.export('mul_f', 'f8(f8, f8)')
def mul(a, b):

    return a*b

cc.compile()

and the setup.py looks something like this

import setuptools
from numpy.distutils.core import setup
from numba_project.src.numba_source import cc

setup(
        name = 'my_numba_examples',
        # other setup arguments,
        ext_modules = [cc.distutils_extension()]
     )

Now, when i build extension modules through the build_ext command, my extension module is built into the src/ folder. i.e.,

- numba_project/
    - numba_project/
        - __init__.py
        - src/
            - numba_source.py
            - my_module.so
        - lib/
    - setup.py

But, I would like to build the extension into the lib/ folder. According to the reference manual, the numba AOT compiler provides the option to specify the output directory name and file name where the extension would be built, through the CC instance attributes output_dir and output_file respectively. But this works only if I compile the code myself by calling the CC function compile(). When the code is compiled by the setup.py, the output_dir and output_file attributes are to be ignored.

So, I currently am not able to specify the target directory for my extension. I tried the Python dotted name (which is the standard while creating a C or a Fortran extension as explained here) while naming the module, i.e., something like

cc = CC('numba_project.lib.my_module')

but I get the following error

ValueError: basename should be a simple module name, not qualified name

Any suggestions on how to build the module under a target directory when building the sources from setup.py would be appreciated. Thanks!

numba python-packaging setup.py distutils python
1 Answers

This one was a little tricky to figure out (like most AOT functions of Numba). This is all via the command line (or put it in your AOT source file), which will compile just fine. If you don't put an output_dir in it will make it in the current location when you use cc.compile()

So first you should create a cc instance like:

from numba .pycc import CC
cc = CC('my_module') # name your module
cc.output_dir = 'C://Username//My_Name//my_output_dir//' # set the output directory

All the CC flags are passed this way for AOT compiling. You can just type this in a terminal before you finally do a:

cc.compile()

And yes I confimed that's where the output PYD goes. You could also put it in your script under a: if __name__ == "__main__": block with cc.compile() if you want to run the script and it will compile at the same time.

user_60371180
Related