Calling C/C++ from Python?

ctypes module is part of the standard library, and therefore is more stable and widely available than swig, which always tended to give me problems.

With ctypes, you need to satisfy any compile time dependency on python, and your binding will work on any python that has ctypes, not just the one it was compiled against.

Suppose you have a simple C++ example class you want to talk to in a file called foo.cpp:

#include <iostream>

class Foo{
    public:
        void bar(){
            std::cout << "Hello" << std::endl;
        }
};

Since ctypes can only talk to C functions, you need to provide those declaring them as extern "C"

extern "C" {
    Foo* Foo_new(){ return new Foo(); }
    void Foo_bar(Foo* foo){ foo->bar(); }
}

Next you have to compile this to a shared library

g++ -c -fPIC foo.cpp -o foo.o
g++ -shared -Wl,-soname,libfoo.so -o libfoo.so  foo.o

And finally you have to write your python wrapper (e.g. in fooWrapper.py)

from ctypes import cdll
lib = cdll.LoadLibrary('./libfoo.so')

class Foo(object):
    def __init__(self):
        self.obj = lib.Foo_new()

    def bar(self):
        lib.Foo_bar(self.obj)

Once you have that you can call it like

f = Foo()
f.bar() #and you will see "Hello" on the screen

You should have a look at Boost.Python. Here is the short introduction taken from their website:

The Boost Python Library is a framework for interfacing Python and C++. It allows you to quickly and seamlessly expose C++ classes functions and objects to Python, and vice-versa, using no special tools -- just your C++ compiler. It is designed to wrap C++ interfaces non-intrusively, so that you should not have to change the C++ code at all in order to wrap it, making Boost.Python ideal for exposing 3rd-party libraries to Python. The library's use of advanced metaprogramming techniques simplifies its syntax for users, so that wrapping code takes on the look of a kind of declarative interface definition language (IDL).

The quickest way to do this is using SWIG.

Example from SWIG tutorial:

/* File : example.c */
int fact(int n) {
    if (n <= 1) return 1;
    else return n*fact(n-1);
}

Interface file:

/* example.i */
%module example
%{
/* Put header files here or function declarations like below */
extern int fact(int n);
%}

extern int fact(int n);

Building a Python module on Unix:

swig -python example.i
gcc -fPIC -c example.c example_wrap.c -I/usr/local/include/python2.7
gcc -shared example.o example_wrap.o -o _example.so

Usage:

>>> import example
>>> example.fact(5)
120

Note that you have to have python-dev. Also in some systems python header files will be in /usr/include/python2.7 based on the way you have installed it.

From the tutorial:

SWIG is a fairly complete C++ compiler with support for nearly every language feature. This includes preprocessing, pointers, classes, inheritance, and even C++ templates. SWIG can also be used to package structures and classes into proxy classes in the target language — exposing the underlying functionality in a very natural manner.

I think cffi for python can be an option.

The goal is to call C code from Python. You should be able to do so without learning a 3rd language: every alternative requires you to learn their own language (Cython, SWIG) or API (ctypes). So we tried to assume that you know Python and C and minimize the extra bits of API that you need to learn.

http://cffi.readthedocs.org/en/release-0.7/

pybind11 minimal runnable example

pybind11 was previously mentioned at https://stackoverflow.com/a/38542539/895245 but I would like to give here a concrete usage example and some further discussion about implementation.

All and all, I highly recommend pybind11 because it is really easy to use: you just include a header and then pybind11 uses template magic to inspect the C++ class you want to expose to Python and does that transparently.

The downside of this template magic is that it slows down compilation immediately adding a few seconds to any file that uses pybind11, see for example the investigation done on this issue. PyTorch agrees. A proposal to remediate this problem has been made at: https://github.com/pybind/pybind11/pull/2445

Here is a minimal runnable example to give you a feel of how awesome pybind11 is:

class_test.cpp

#include <string>

#include <pybind11/pybind11.h>

struct ClassTest {
    ClassTest(const std::string &name, int i) : name(name), i(i) { }
    void setName(const std::string &name_) { name = name_; }
    const std::string getName() const { return name + "z"; }
    void setI(const int i) { this->i = i; }
    const int getI() const { return i + 1; }
    std::string name;
    int i;
};

namespace py = pybind11;

PYBIND11_PLUGIN(class_test) {
    py::module m("my_module", "pybind11 example plugin");
    py::class_<ClassTest>(m, "ClassTest")
        .def(py::init<const std::string &, int>())
        .def("setName", &ClassTest::setName)
        .def("getName", &ClassTest::getName)
        .def_readwrite("name", &ClassTest::name)
        .def("setI", &ClassTest::setI)
        .def("getI", &ClassTest::getI)
        .def_readwrite("i", &ClassTest::i);
    return m.ptr();
}

class_test_main.py

#!/usr/bin/env python3

import class_test

my_class_test = class_test.ClassTest("abc", 1);
print(my_class_test.getName())
print(my_class_test.getI())
my_class_test.setName("012")
my_class_test.setI(2)
print(my_class_test.getName())
print(my_class_test.getI())
assert(my_class_test.getName() == "012z")
assert(my_class_test.getI() == 3)

Compile and run:

#!/usr/bin/env bash
set -eux
sudo apt install pybind11-dev
g++ `python3-config --cflags` -shared -std=c++11 -fPIC class_test.cpp \
  -o class_test`python3-config --extension-suffix` `python3-config --libs`
./class_test_main.py

Stdout output:

abcz
2
012z
3

If we tried to use a wrong type as in:

my_class_test.setI("abc")

it blows up as expected:

Traceback (most recent call last):
  File "/home/ciro/test/./class_test_main.py", line 9, in <module>
    my_class_test.setI("abc")
TypeError: setI(): incompatible function arguments. The following argument types are supported:
    1. (self: my_module.ClassTest, arg0: int) -> None

Invoked with: <my_module.ClassTest object at 0x7f2980254fb0>, 'abc'

This example shows how pybind11 allows you to effortlessly expose the ClassTest C++ class to Python!

Notably, Pybind11 automatically understands from the C++ code that name is an std::string, and therefore should be mapped to a Python str object.

Compilation produces a file named class_test.cpython-36m-x86_64-linux-gnu.so which class_test_main.py automatically picks up as the definition point for the class_test natively defined module.

Perhaps the realization of how awesome this is only sinks in if you try to do the same thing by hand with the native Python API, see for example this example of doing that, which has about 10x more code: https://github.com/cirosantilli/python-cheat/blob/4f676f62e87810582ad53b2fb426b74eae52aad5/py_from_c/pure.c On that example you can see how the C code has to painfully and explicitly define the Python class bit by bit with all the information it contains (members, methods, further metadata...). See also:

• Can python-C++ extension get a C++ object and call its member function?
• Exposing a C++ class instance to a python embedded interpreter
• A full and minimal example for a class (not method) with Python C Extension?
• Embedding Python in C++ and calling methods from the C++ code with Boost.Python
• Inheritance in Python C++ extension

pybind11 claims to be similar to Boost.Python which was mentioned at https://stackoverflow.com/a/145436/895245 but more minimal because it is freed from the bloat of being inside the Boost project:

pybind11 is a lightweight header-only library that exposes C++ types in Python and vice versa, mainly to create Python bindings of existing C++ code. Its goals and syntax are similar to the excellent Boost.Python library by David Abrahams: to minimize boilerplate code in traditional extension modules by inferring type information using compile-time introspection.

The main issue with Boost.Python—and the reason for creating such a similar project—is Boost. Boost is an enormously large and complex suite of utility libraries that works with almost every C++ compiler in existence. This compatibility has its cost: arcane template tricks and workarounds are necessary to support the oldest and buggiest of compiler specimens. Now that C++11-compatible compilers are widely available, this heavy machinery has become an excessively large and unnecessary dependency.

Think of this library as a tiny self-contained version of Boost.Python with everything stripped away that isn't relevant for binding generation. Without comments, the core header files only require ~4K lines of code and depend on Python (2.7 or 3.x, or PyPy2.7 >= 5.7) and the C++ standard library. This compact implementation was possible thanks to some of the new C++11 language features (specifically: tuples, lambda functions and variadic templates). Since its creation, this library has grown beyond Boost.Python in many ways, leading to dramatically simpler binding code in many common situations.

pybind11 is also the only non-native alternative hightlighted by the current Microsoft Python C binding documentation at: https://docs.microsoft.com/en-us/visualstudio/python/working-with-c-cpp-python-in-visual-studio?view=vs-2019 (archive).

Tested on Ubuntu 18.04, pybind11 2.0.1, Python 3.6.8, GCC 7.4.0.

I love cppyy, it makes it very easy to extend Python with C++ code, dramatically increasing performance when needed.

It is powerful and frankly very simple to use,

here it is an example of how you can create a numpy array and pass it to a class member function in C++.

cppyy_test.py

import cppyy
import numpy as np
cppyy.include('Buffer.h')


s = cppyy.gbl.Buffer()
numpy_array = np.empty(32000, np.float64)
s.get_numpy_array(numpy_array.data, numpy_array.size)
print(numpy_array[:20])

Buffer.h

struct Buffer {
  void get_numpy_array(double *ad, int size) {
    for( long i=0; i < size; i++)
        ad[i]=i;
  }
};

You can also create a Python module very easily (with CMake), this way you will avoid recompile the C++ code all the times.