How to make a general function wrapper pickable with pybind11?

Viewed 673

I have a class in c++ containing among other things a std::function. Via pybind11 I created a binding for that class, which worked fine. Now I would like to make the class pickleable. However, when trying to pickle an instance of the class in python, I get a TypeError: can't pickle PyCapsule objects.

To be precise, here is a minimal example: the c++ class looks like

#include<functional>

using Function = std::function<double(double)>;

struct Test {
    Function func;
};

The code to generate the python binding reads

#include "pybind11/pybind11.h"
#include "pybind11/functional.h"


namespace py = pybind11;


PYBIND11_MODULE(my_module, m) {
    py::class_<Test>(m, "Test")
        .def(py::init<Function>())
        .def(py::pickle(
                    [](const Test& t)
                    {
                        return py::make_tuple(t.func);
                    },
                    [](py::tuple tup)
                    {
                        return Test{tup[0].cast<Function>()};
                    }
                    ));
}

Now (after the above has been built), if one tries to pickle an instance of the class in python

import pickle

from my_module import Test


def identity(entity):
    return entity


if __name__ == "__main__":
    test = Test(identity)
    with open('test.pickle', 'wb') as out_file:
        pickle.dump(test, out_file)

one gets the error mentioned above:

TypeError: can't pickle PyCapsule objects

which is raised by the line pickle.dump(test, out_file).

Is there any way to make pickling work without loosing generality? That is, I would really like to be able to invoke the class with an arbitrary python callable with signature float -> float.

One gets the same error if func is wrapped into a py::cpp_function, that is, if one replaces return py::make_tuple(t.func) with return py::make_tuple(py::cpp_function{t.func}).

0 Answers
Related