Is there a way to pickle objects so that in separate run to retrieve them

user_15375560 0
Viewed 173

I wonder if it is possible somehow to read the pickle file correctly without providing MyClass code.

import pickle


class MyClass:
    def __init__(self, n):
        self._n = n

    def give_vec(self):
        return [1 for _ in range(self._n)]


if __name__ == '__main__':
    m = MyClass(30)

    with open('test.pickle', 'wb') as p:
        pickle.dump(m, p)

Assume we have run the code above and now start a python interpreter and try the following

>>> import pickle
>>> p = open('test.pickle', 'rb')
>>> a = pickle.load(p)

So that we won't get this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: Can't get attribute 'MyClass' on <module '__main__' (built-in)>

The idea is to provide a pickle that has some interface without specifying the exact class

pickle python-3.x serialization python
1 Answers

Unfortunately, it is not possible with pickle alone.

From https://docs.python.org/3/library/pickle.html#pickle-picklable:

Neither the function’s code, nor any of its function attributes are pickled. Thus the defining module must be importable in the unpickling environment, and the module must contain the named object, otherwise an exception will be raised.

So it is only possible to pickle data, not the behavior of the associated classes.

But you can use dill:

dill can be used to store python objects to a file, but the primary usage is to send python objects across the network as a byte stream. dill is quite flexible, and allows arbitrary user defined classes and functions to be serialized.

Just replace import pickle with import dill as pickle, and it will work:

import dill as pickle


class MyClass:
    def __init__(self, n):
        self._n = n

    def give_vec(self):
        return [1 for _ in range(self._n)]


if __name__ == '__main__':
    m = MyClass(30)

    with open('test.pickle', 'wb') as p:
        pickle.dump(m, p)

>>> import dill as pickle
>>> p = open('test.pickle', 'rb')
>>> a = pickle.load(p)
>>> a
<__main__.MyClass object at 0x7eff33758340>
user_30128840
Related