Caching class attributes in Python

Viewed 59507

I'm writing a class in python and I have an attribute that will take a relatively long time to compute, so I only want to do it once. Also, it will not be needed by every instance of the class, so I don't want to do it by default in __init__.

I'm new to Python, but not to programming. I can come up with a way to do this pretty easily, but I've found over and over again that the 'Pythonic' way of doing something is often much simpler than what I come up with using my experience in other languages.

Is there a 'right' way to do this in Python?

11 Answers

3.8 ≤ Python @property and @functools.lru_cache have been combined into @cached_property.

import functools
class MyClass:
    @functools.cached_property
    def foo(self):
        print("long calculation here")
        return 21 * 2

3.2 ≤ Python < 3.8

You should use both @property and @functools.lru_cache decorators:

import functools
class MyClass:
    @property
    @functools.lru_cache()
    def foo(self):
        print("long calculation here")
        return 21 * 2

This answer has more detailed examples and also mentions a backport for previous Python versions.

Python < 3.2

The Python wiki has a cached property decorator (MIT licensed) that can be used like this:

import random
# the class containing the property must be a new-style class
class MyClass(object):
   # create property whose value is cached for ten minutes
   @cached_property(ttl=600)
   def randint(self):
       # will only be evaluated every 10 min. at maximum.
       return random.randint(0, 100)

Or any implementation mentioned in the others answers that fits your needs.
Or the above mentioned backport.

Python 3.8 includes the functools.cached_property decorator.

Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance. Similar to property(), with the addition of caching. Useful for expensive computed properties of instances that are otherwise effectively immutable.

This example is straight from the docs:

from functools import cached_property

class DataSet:
    def __init__(self, sequence_of_numbers):
        self._data = sequence_of_numbers

    @cached_property
    def stdev(self):
        return statistics.stdev(self._data)

    @cached_property
    def variance(self):
        return statistics.variance(self._data)

The limitation being that the object with the property to be cached must have a __dict__ attribute that is a mutable mapping, ruling out classes with __slots__ unless __dict__ is defined in __slots__.

As mentioned, functools.cached_property will work for cached instance attributes. For cached class attributes:

from functools import cache

class MyClass:
    @classmethod
    @property
    @cache
    def foo(cls):
        print('expensive calculation')
        return 42
>>> MyClass.foo
expensive calculation
42
>>> MyClass.foo
42

And if you want a reusable decorator:

def cached_class_attr(f):
    return classmethod(property(cache(f)))

class MyClass:
    @cached_class_attr
    def foo(cls):
        ...

The dickens package (not mine) offers cachedproperty, classproperty and cachedclassproperty decorators.

To cache a class property:

from descriptors import cachedclassproperty

class MyClass:
    @cachedclassproperty
    def approx_pi(cls):
        return 22 / 7

Most if not all current answers are about caching instance attributes. To cache class attributes, you can simply use a dictionary. This ensures the attributes are calculated once per class, instead of once per instance.

mapping = {}

class A:
    def __init__(self):
        if self.__class__.__name__ not in mapping:
            print('Expansive calculation')
            mapping[self.__class__.__name__] = self.__class__.__name__
        self.cached = mapping[self.__class__.__name__]

To illustrate,

foo = A()
bar = A()
print(foo.cached, bar.cached)

gives

Expansive calculation
A A

With Python 2, but not Python 3, here's what I do. This is about as efficient as you can get:

class X:
    @property
    def foo(self):
        r = 33
        self.foo = r
        return r

Explanation: Basically, I'm just overloading a property method with the computed value. So after the first time you access the property (for that instance), foo ceases to be a property and becomes an instance attribute. The advantage of this approach is that a cache hit is as cheap as possible because self.__dict__ is being used as the cache, and there is no instance overhead if the property is not used.

This approach doesn't work with Python 3.

Related