Packaging a Python extension for a C++ library: what to include, how to include it in .whl?

Question

I'm writing an extension for a C++ library to make it available in Python using Pybind11. The library itself depends on a couple of other C++ libraries.

What I don't get is what files am I supposed to include in my distribution package and how. After mixing up some code from Python Packaging Guide and Building C++ Extensions I got the following files

setup.py


from setuptools import setup, Extension
#from distutils.core import setup, Extension #used this at first, switched to setuptools. Didn't see the difference

src = ['module.cpp', /*... other cpp files */]
include = ['MyLibrary/include', /*... other header files for 3rdparty libs*/]


module = Extension(
    'TestlibPy', 
    sources = src,
    include_dirs= include,
    libraries=[/* library names*/]
    lib_dirs=[/*library dirs*/]
    language='c++',
    )

setup(
    ext_modules = [module],
)

setup.cfg

[metadata]
name = TestlibPy
version = 0.0.1
description = Python interface for Test library
classifiers =
    Programming Language:: Python :: 3

[options]
packages = find:
python_requires = >=3.6

pyproject.toml

[build-system]
requires = [
   "setup tools>=42",
   "wheel"

]
build-backend = "setuptools.build_meta"

After building with

py -m build

I got the bare minimum packages that don't even include the headers (and I don't get the logic here - it builds the distribution with them in mind, from the directory provided in 'include', yet doesn't think other users will need those heades?).

So I wrote a MANIFEST.in:

graft MyLibC++/include
graft MyLib/3rdpartyLibs

After another build I get package.tar.gz - contains everything I asked for and works on my other laptop after installation, but it's an unencoded archive that gives away the source code to anyone who bothers to look (I obviously don't want that); package.whl - ignores my MANIFEST.in and doesn't seem to work (is it supposed to? Did all the necessary information get into the .pyd file without me knowing any better?)

My questions are:

Is it alright to include all the 3rd party C++ libraries + pybind11 in my distribution package, or is there some better way to do things?
Should the C++ library be in .dll format, or can I get away with a bunch of .cpp files?
Can I somehow write the .hpp and 3rdparty files to the .whl package? Or it should work without them?

That's my first time working with Python, extensions and packages, so maybe I'm asking all the wrong questions. Any advice would be helpful.

Packaging a Python extension for a C++ library: what to include, how to include it in .whl?

0 Answers