Best way to register all subclasses

Question

I am currently developing a piece of software where the I have class instamces that are generated from dictionaries. The way these dictionariea file are structured is as follows:

layer_dict = {
    "layer_type": "Conv2D",
    "name": "conv1",
    "kernel_size": 3,
    ...
}

Then, the following code is ran

def create_layer(layer_dict):
    LayerType = getattr(layers, layer_dict['layer_type']
    del layer_dict['layer_type']
    return LayerType(**layer_dict)

Now, I want to support the creation of new layer types (by subclassing the BaseLayer class). I've thought of a few ways to do this and thought I'd ask which way is best and why as I don't have much experience developing software (finishing an MSc in comp bio).

Method 1: Metaclasses The first method I thought of was to have a metaclass that registers every subclass of BaseLayer in a dict and do a simple lookup of this dict instead of using getattr.

class MetaLayer(type)
    layers = {}
    def __init__(cls, name, bases, dct):
        if name in MetaLayer.layers:
            raise ValueError('Cannot have more than one layer with the same name')
        MetaLayer.layers[name] = cls

Benefit: The metaclass can make sure that no two classes have the same name. The user doesn't need to think about anything but subclassing when creating new layers.
Downside: Metaclasses are difficult to understand and often frowned upon

Method 2: Traversing the __subclasses__ tree

The second method I thought of was to use the __subclassess__ function of BaseLayer to get a list of all subclasses, then create a dict with Layer.__name__ as keys and Layer as values. See example code below:

def get_subclasses(cls):
    """Returns all classes that inherit from `cls`
    """
    subclasses = {
        sub.__name__: sub for sub in cls.__subclasses__()
    }

    subsubclasses = (
        get_subclasses(sub) for sub in subclasses.values()
    )
    subsubclasses = {
        name: sub for subs in subsubclasses for name, sub in subs.items()
    }
    return {**subclasses, ** subsubclasses}

Benefit: Easy to explain how this works.
Downside: We might end up with two layers having the same name.

Method 3: Using a class decorator The final method is my favourite as it doesn't hide any implementation details in a metaclass, and still manages to prevent multiple classes with the same name.

Here the layers module has a global variable named layers and a decorator named register_layer, which simply adds the decorated classes to the layers dict. See code below.

layers = {}
def register_layer(cls):
    if cls.__name__ in layers:
        raise ValueError('Cannot have two layers with the same name')
    layers[cls.__name__] = cls
    return cls

Benefit: No metaclasses and no way of having two layers with the same name.
Downside: Requires a global variable, which is often frowned upon.

So, my question is, which method is preferable? And more importantly, why?

user_108205 · Answer

Actually - that is the kind of things metaclases are designed for. As you can see from the options you stated above, it is the simpler and more straightforward design.

They are sometimes "frowned upon" because of two things: (1) people don't understand then and don't care for understanding; (2) people misuse then when they are actually not needed; (3) they are hard to combine - so if any of your classes is to be used with a mixn that have a different metaclass (say abc.ABC), you have also to produce a combining metaclass.

Method 4: `__init_subclass__`

Now, that said, from Python 3.6, there is a new feature that can cover your usecase without the need for metaclasses: the class __init_subclass__ method: it is called as a classmethod on the base class when subclasses of it are created.

All you need is to write a proper __init_subclass__ method on your BaseLayer class and have all the benefits you'd have from the implementation in the metaclasses and none of the downsides

user_6890912 · Answer

Like you, I like the class decorator approach as it is more readable.

You can avoid using a global variable by making the class decorator itself a class, and making layers a class variable instead. You can also avoid possible name collision by joining the target class' name with its module name:

class register_layer:
    layers = {}
    def __new__(cls, target):
        cls.layers['.'.join((target.__module__, target.__name__))] = target
        return target

Best way to register all subclasses

2 Answers

Method 4: __init_subclass__

Method 4: `__init_subclass__`