The implementation of lru_cache in functools only support hashable arguments.
I'm trying to make it work for any structure with a __ str __ method implementation by combining 3 decorators in a single one: One that convert the structure to string (hashable data), one for the cache using the string, and one that recovers the data structure from the string data. I don't like the last decorator, since it requires remaking the whole data from a string, which can be computationally expensive. It could also be the case that the structure can't be recovered from the string version with a literal_eval operation.
Is there any better way to refactor this so it uses the original input data, rather than recovering it from a string? If there is an out of the box implementation for this, that would be helpful too.
from functools import lru_cache
from ast import literal_eval
def data_2_str(f):
def wrapper(*data):
string_data = str(data)
return f(string_data)
return wrapper
def str_2_data(f):
def wrapper(string_data):
data = literal_eval(string_data)
return f(*data)
return wrapper
def string_lru_cache(maxsize:int=128):
def wrapper1(f):
@data_2_str
@lru_cache(maxsize=maxsize)
@str_2_data
def wrapper2(*data):
return f(*data)
return wrapper2
return wrapper1
if __name__ == "__main__":
from time import sleep, time
from typing import List
class Pipeline:
def __init__(self):
@string_lru_cache(maxsize=2)
def operation( numbers:List[float])->float:
sleep(2)
return sum(numbers)
self.operation = operation
p = Pipeline()
t0 = time()
p.operation([1,2,3,4,5])
print("operation time:", time()-t0)
t0 = time()
p.operation([1,2,3,4,5])
print("operation time:", time()-t0)
$ python utils_cache.py
operation time: 2.004383087158203
operation time: 2.3126602172851562e-05