Python: how does Pickle work with defaultdict

user_4574182 0
Viewed 530

I am new to Python and am playing with Pickle and don't understand how this works

I define a defaultdict, write it to pickle. Then in a different script I read it and it still behaves like a defaultdict even without importing collections

script1:

import pickle
from collections import defaultdict

x = defaultdict(list)

x['a'].append(1)
print(x)

with open('pick','wb') as f:
    pickle.dump( x, f )

script2:

import pickle

with open('pick','rb') as f:
    x = pickle.load( f )

x['b'].append(2)
print(x)

y = dict()

try:
    y['b'].append(2)
    print(y)
except KeyError:
    print("Can't append to y")

running:

$ python3 pick2.py
defaultdict(<class 'list'>, {'a': [1], 'b': [2]}) 
Can't append to y

So, the 2nd script doesn't import defaultdict but the pickled x still acts like one. I'm confused :)

How does this work in Python? Thanks for any info :)

defaultdict pickle python-3.x python
1 Answers

First of all, if you look at the pickle docs, specifically:

pickle can save and restore class instances transparently, however the class definition must be importable and live in the same module as when the object was stored

So what this is telling us is that pickle will import the module that defines the object you are unpickling.

We can show this with a small example, consider the following folder structure:

parent/
|-- a.py
|-- sub/

sub is an empty sub-folder
a.py holds an example class

# a.py
class ExampleClass:
    def __init__(self):
        self.var = 'This is a string'

Now starting the python console in the parent directory:

alex@toaster:parent$ python3
>>> import pickle
>>> from a import ExampleClass
>>> x = ExampleClass()
>>> x.var
'This is a string'
>>> with open('eg.p', 'wb') as f:
...     pickle.dump(x, f)

Exit the shell. Move to the sub directory and try to load the pickled ExampleClass object.

alex@toaster:sub$ python3
>>> import pickle
>>> with open('../eg.p', 'rb') as f:
...     x = pickle.load(f)
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ModuleNotFoundError: No module named 'a'

We get a ModuleNotFoundError as pickle cannot load the class definition from the module a (it's in a different directory). In your case, python can load the collections.defaultdict class as this module is on the PYTHONPATH. However, to continue to use the module(s) imported by pickle you will still need to import them yourself; eg you want to create another defaultdict in script2.py.

To find out more about modules look here, specifically 6.1.2 The Module Search Path.

user_32797160
Related