How can I pickle a dynamically created nested class in python?

mixedpickles_49628 0
Viewed 24687

I have a nested class:

class WidgetType(object):
    
    class FloatType(object):
        pass
    
    class TextType(object):
        pass

.. and an object that refers the nested class type (not an instance of it) like this

class ObjectToPickle(object):
     def __init__(self):
         self.type = WidgetType.TextType

Trying to serialize an instance of the ObjectToPickle class results in:

PicklingError: Can't pickle <class 'setmanager.app.site.widget_data_types.TextType'>

Is there a way to pickle nested classes in python?

pickle class nested python
7 Answers

The pickle module is trying to get the TextType class from the module. But since the class is nested it doesn't work. jasonjs's suggestion will work. Here are the lines in pickle.py responsible for the error message:

    try:
        __import__(module)
        mod = sys.modules[module]
        klass = getattr(mod, name)
    except (ImportError, KeyError, AttributeError):
        raise PicklingError(
            "Can't pickle %r: it's not found as %s.%s" %
            (obj, module, name))

klass = getattr(mod, name) will not work in the nested class case of course. To demonstrate what is going on try to add these lines before pickling the instance:

import sys
setattr(sys.modules[__name__], 'TextType', WidgetType.TextType)

This code adds TextType as an attribute to the module. The pickling should work just fine. I don't advice you to use this hack though.

user_978280

In Sage (www.sagemath.org), we have many instances of this pickling issue. The way we decided to systematically solve it is to put the outer class inside a specific metaclass whose goal is to implement and hide the hack. Note that this automatically propagate through nested classes if there are several level of nesting.

user_2876820

Pickle only works with classes defined in module scope (top level). In this case, it looks like you could define the nested classes in module scope and then set them as properties on WidgetType, assuming there's a reason not to just reference TextType and FloatType in your code. Or, import the module they're in and use widget_type.TextType and widget_type.FloatType.

user_1627680

Nadia's answer is pretty complete - it is practically not something you want to be doing; are you sure you can't use inheritance in WidgetTypes instead of nested classes?

The only reason to use nested classes is to encapsulate classes working together closely, your specific example looks like an immediate inheritance candidate to me - there is no benefit in nesting WidgetType classes together; put them in a module and inherit from the base WidgetType instead.

user_2236320

This seems to work fine in newer versions of Python. I tried it in v3.8 and it was able to pickle and unpickle the nested class.

user_38879400
Related