Serialize a function of a class without serializing the class itself

FilterFeeder 0
Viewed 29

I'm using pyro5, and I want to make a remote object function as a worker class, where a client can ask it to perform various tasks by sending it functions to execute.

For this to work, I need to serialize functions, but the serializers available through Pyro5 does not support serializing functions.

Instead, I intend to use Dill, and somehow send the already serialized function through Pyro5.

However, when trying to serialize a function, I receive errors which say that I can't serialize X, where X is the enclosing class where the function lives. How can I prevent this from happening?

Actually, I was not able to reproduce this in a small script with a class and a function. I am running the dill serializer in a Qt app, not sure if that's the problem?

Alternatively, does anyone have a better idea on how to approach this?

dill
1 Answers

I'm the dill author. There are several ways to approach this that may work for you. It's hard to tell without more information on your class in question, so I'll have to make assumptions.

If assume you are working with a class that you are importing from an installed package, then you can do something like this:

>>> # Mock a unserializable builtin class "Foo"
>>> import ctypes
>>> PyObjectType = ctypes.py_object(lambda :None)
>>> class Foo(PyObjectType.__class__):
...   def __init__(self, x):
...     self.x = x
...   def bar(self, y):
...     return y + self.x
... 
>>> f = Foo(1)
>>> f.bar(2)
3
>>> # serializing the method fails
>>> # dill.dumps(f.bar) #FAILS
>>> # so we create a wrapping function
>>> bar = lambda x:f.bar(x)
>>> 
>>> import dill
>>> dill.dumps(bar)
b'\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_create_code\nq\x01(K\x01K\x00K\x01K\x03KCC\nt\x00\xa0\x01|\x00\xa1\x01S\x00q\x02N\x85q\x03X\x01\x00\x00\x00fq\x04X\x03\x00\x00\x00barq\x05\x86q\x06X\x01\x00\x00\x00xq\x07\x85q\x08X\x07\x00\x00\x00<stdin>q\tX\x08\x00\x00\x00<lambda>q\nK\x01C\x00q\x0b))tq\x0cRq\rc__builtin__\n__main__\nh\nNNtq\x0eRq\x0f}q\x10}q\x11X\x0f\x00\x00\x00__annotations__q\x12}q\x13s\x86q\x14b.'

This works, as long as the class is available for import wherever you are going to use dill.loads to restore the function.

Or if you need to ship the class as well, you could also create a wrapping class (i.e. a derived class) that includes a __reduce__ method which informs dill how to serialize the class. https://docs.python.org/3/library/pickle.html#object.__reduce__

There are other approaches as well, but it would help to have more information on what you are dealing with.

Mike McKerns0
Related