Pandas 1.0+ cannot pickle.load dict containing DataFrame with MultiIndex

Question

I store related DataFrames, each with a MultiIndex, in one dict and pickle it. In Pandas 0.22-0.25 I could pickle.load the dict. Starting in Pandas 1.0 I cannot: I get an AttributeError. Judging from the Pandas 1.0 "What's New" page, it might be because frozen.py was deprecated. Here is reproduction code:

import pickle
import pandas as pd
import numpy as np

# Create DataFrames and dict
df1 = pd.DataFrame(data=np.ones((2, 2)))
df2 = pd.DataFrame(data=2*np.ones((2, 2)), index=pd.MultiIndex.from_arrays([[1, 2], ['a', 'a']]))
dict12 = {'df1':df1, 'df2':df2}

# Pickle the dict and DataFrames
with open('dict12.pkl', 'wb') as f:
    pickle.dump(dict12, f)
df1.to_pickle('df1.pkl')
df2.to_pickle('df2.pkl')

# Read DataFrames individually
df1 = pd.read_pickle("df1.pkl")
df2 = pd.read_pickle("df2.pkl")

# Read pickled dict
with open("dict12.pkl", "rb") as f:
    dict12 = pickle.load(f)

On Pandas 0.25, the df1, df2, and dict12 all are read/loaded. But on Pandas 1.0, the df1 and df2 read in, but the dict12 is not loaded and I get the error:

AttributeError: Can't get attribute 'FrozenNDArray' on

The error does not depend on the dict having multiple keys. If instead we have

dict12 = {'df2': df2}
with open('dict12.pkl', 'wb') as f:
    pickle.dump(dict12, f)
with open("dict12.pkl", "rb") as f:
    dict12 = pickle.load(f)

the same error occurs. Meanwhile, when the pickled dict has a DataFrame without a MultiIndex, there is no problem

dict12 = {'df1': df1}
with open('dict12.pkl', 'wb') as f:
    pickle.dump(dict12, f)
with open("dict12.pkl", "rb") as f:
    dict12 = pickle.load(f)

How can I still work with these pickled dict in Pandas 1.0+?

Update

The above is true if

We use pickle.dump on the dict in Pandas 0.25-
We use pickle.load on the dict in Pandas 1.0+

However, things work fine if we both pickle.dump and pickle.load in the same Pandas (0. or 1.)

So really the question is how to pickle.load a dict containing a DataFrame with MultiIndex in Pandas 1.0+, when the dict was pickled in Pandas 0.25-?

Pandas 1.0+ cannot pickle.load dict containing DataFrame with MultiIndex

Update

0 Answers