Using the __call__ method of a metaclass instead of __new__?

eliben_8206 0
Viewed 29493

When discussing metaclasses, the docs state:

You can of course also override other class methods (or add new methods); for example defining a custom __call__() method in the metaclass allows custom behavior when the class is called, e.g. not always creating a new instance.

[Editor's note: This was removed from the docs in 3.3. It's here in 3.2: Customizing class creation]

My questions is: suppose I want to have custom behavior when the class is called, for example caching instead of creating fresh objects. I can do this by overriding the __new__ method of the class. When would I want to define a metaclass with __call__ instead? What does this approach give that isn't achievable with __new__?

metaclass oop python
6 Answers

I thought a fleshed out Python 3 version of pyroscope's answer might be handy for someone to copy, paste and hack about with (probably me, when I find myself back at this page looking it up again in 6 months). It is taken from this article:

class Meta(type):

     @classmethod
     def __prepare__(mcs, name, bases, **kwargs):
         print('  Meta.__prepare__(mcs=%s, name=%r, bases=%s, **%s)' % (
             mcs, name, bases, kwargs
         ))
         return {}

     def __new__(mcs, name, bases, attrs, **kwargs):
         print('  Meta.__new__(mcs=%s, name=%r, bases=%s, attrs=[%s], **%s)' % (
             mcs, name, bases, ', '.join(attrs), kwargs
         ))
         return super().__new__(mcs, name, bases, attrs)

     def __init__(cls, name, bases, attrs, **kwargs):
         print('  Meta.__init__(cls=%s, name=%r, bases=%s, attrs=[%s], **%s)' % (
             cls, name, bases, ', '.join(attrs), kwargs
         ))
         super().__init__(name, bases, attrs)

     def __call__(cls, *args, **kwargs):
         print('  Meta.__call__(cls=%s, args=%s, kwargs=%s)' % (
             cls, args, kwargs
         ))
         return super().__call__(*args, **kwargs)

print('** Meta class declared')

class Class(metaclass=Meta, extra=1):

     def __new__(cls, myarg):
         print('  Class.__new__(cls=%s, myarg=%s)' % (
             cls, myarg
         ))
         return super().__new__(cls)

     def __init__(self, myarg):
         print('  Class.__init__(self=%s, myarg=%s)' % (
             self, myarg
         ))
         self.myarg = myarg
         super().__init__()

     def __str__(self):
         return "<instance of Class; myargs=%s>" % (
             getattr(self, 'myarg', 'MISSING'),
         )

print('** Class declared')

Class(1)
print('** Class instantiated')

Outputs:

** Meta class declared
  Meta.__prepare__(mcs=<class '__main__.Meta'>, name='Class', bases=(), **{'extra': 1})
  Meta.__new__(mcs=<class '__main__.Meta'>, name='Class', bases=(), attrs=[__module__, __qualname__, __new__, __init__, __str__, __classcell__], **{'extra': 1})
  Meta.__init__(cls=<class '__main__.Class'>, name='Class', bases=(), attrs=[__module__, __qualname__, __new__, __init__, __str__, __classcell__], **{'extra': 1})
** Class declared
  Meta.__call__(cls=<class '__main__.Class'>, args=(1,), kwargs={})
  Class.__new__(cls=<class '__main__.Class'>, myarg=1)
  Class.__init__(self=<instance of Class; myargs=MISSING>, myarg=1)
** Class instantiated

Another great resource highlighted by the same article is David Beazley's PyCon 2013 Python 3 Metaprogramming tutorial.

user_13089670

In the particular example given in the question, overriding __call__ in the metaclass is just superior to overriding __new__ in the class.

suppose I want to have custom behavior when the class is called, for example caching instead of creating fresh objects

  1. If the purpose of caching is efficiency, then caching __new__'s result is not optimal because __init__ is executed anyhow (Data Model: Basic Customization). For example:

    from functools import lru_cache

class MyClass:

    @lru_cache(maxsize=None)
    def __new__(cls, *args, **kwargs):
        return super().__new__(cls)

    def __init__(self, ...)
        "Always executed. Even on cache hit."

  2. Caching __new__'s result is sound only if __init__ has no noticeable effect on an already initialized instance. Otherwise the execution of __init__ on a cached object could lead to annoying side-effects on its other references.

  3. Caching at the metaclass level avoids both the performance issue of 1. and the correctness issue of 2. For example:

    from functools import lru_cache

class CachedInstances(type):

    @lru_cache(maxsize=None)
    def __call__(cls, *args, **kwargs):
        return super().__call__(*args, **kwargs)

class MyClass(metaclass=CachedInstances):

    def __init__(self, ...)
        "Only executed on cache miss."

Note that Michael Ekoka's answer already mention undesired effects that may arise due to repeated __init__ execution within the override __new__ approach (as in my item 2).

user_135440570
Related