Treat distinct argument patterns while using lru_cache decorator as the same calls

user_13375075 0
Viewed 96

As mentioned in the official documentation, Python's functools.lru_cache decorator interprets distinct args patterns as completely different cache keys. For example:

import functools

@functools.lru_cache(maxsize=128)
def test(a, b, *, c, d):
    print(f'Hello from function: ({a}, {b}, {c}, {d})')

test(1, 2, c=42, d='answer')
test(1, 2, d='answer', c=42)
test(b=2, a=1, c=42, d='answer')

While effectively all function calls are the same, the code above produces the following output:

Hello from function: (1, 2, 42, answer)
Hello from function: (1, 2, 42, answer)
Hello from function: (1, 2, 42, answer)

What is the more "pythonic" way to overcome that to treat such kinds of calls as the same?

functools python
2 Answers

Normalize the argument patterns with inspect.signature in another decorator:

import functools
import inspect

def normalize_arg_patterns(func):
    sig = inspect.signature(func)
    @functools.wraps(func)
    def _func(*args, **kwargs):
        ba = sig.bind(*args, **kwargs)
        args, kwargs = ba.args, ba.kwargs
        return func(*args, **kwargs)
    return _func

Usage:

@normalize_arg_patterns  # Add this
@functools.lru_cache(maxsize=128)
def test(a, b, *, c, d):
    print(f'Hello from function: ({a}, {b}, {c}, {d})')

test(1, 2, c=42, d='answer')
test(1, 2, d='answer', c=42)
test(b=2, a=1, c=42, d='answer')

Python 3.2 and 2.7

Normalize the argument patterns with inspect.getcallargs in another decorator:

import functools
import inspect

def normalize_arg_patterns(func):
    f = inspect.unwrap(func)
    @functools.wraps(func)
    def _func(*args, **kwargs):
        args, kwargs = (), inspect.getcallargs(f, *args, **kwargs)
        kwargs = dict(sorted(kwargs.items()))
        return func(*args, **kwargs)
    return _func
user_86017600

I am not very familiar with python, a method I can think of is add another decorator to sort arguments passed to lru_cache.

#!/usr/bin/env python3

from functools import update_wrapper, lru_cache
import inspect

def sorted_params(func):
    sig = inspect.signature(func, follow_wrapped=True)
    def wrapper(*args, **kwargs):
        b = sig.bind(*args, **kwargs)
        return func(*b.args, **b.kwargs)
    return update_wrapper(wrapper, func)

@lru_cache
def test1(a, b, *, c, d):
    print(f'Hello from test1: ({a}, {b}, {c}, {d})')

@sorted_params
@lru_cache
def test2(a, b, *, c, d):
    print(f'Hello from test2: ({a}, {b}, {c}, {d})')

for func in [test1, test2]:
    print('-'*30)
    func(1, 2, c=42, d='answer')
    func(1, 2, d='answer', c=42)
    func(b=2, a=1, c=42, d='answer')

output:

------------------------------
Hello from test1: (1, 2, 42, answer)
Hello from test1: (1, 2, 42, answer)
Hello from test1: (1, 2, 42, answer)
------------------------------
Hello from test2: (1, 2, 42, answer)
user_51898990
Related