why python3 lru_cache doesn't offer an put and query method?

user_8380171 0
Viewed 206

here is my situation, the is_exist function has performance matter.

def is_exist(link :str) -> bool:
    if query_db(link) is True:
         return True
    return False

def process(link: str)
   if is_exist(link) is True:
       return
   # do something
   # put result to database

LRU Cache is a good solution, but after struggle hours i found lru_cache doesn't meet my requirments. What i want like this:

def is_exist(link :str) -> bool:
    if query_db(link) is True:
         return True
    return False

def process(link: str)
   if lru_cache.query(link) is True:
      return

   if is_exist(link) is True:
       lru_cache.add(link)
       return
   # do something
   # put result to database

The LRU Cache in functiontools is a decorator. It doesn't have a query method. If I use @lru_cache decorates is_exist some link will be processed repeatedly when is_exist return false.

PS: put result to db is async

lru caching python-3.x python
3 Answers

functools.lru_cache only caches results but not exceptions. This means functions should raise an exception in any situation that must be re-evaluated later on.

@lru_cache
def is_exist(link: str) -> bool:
    if query_db(link):
         return True   # added to cache
    raise LookupError  # ignored by cache

This allows the client function to check – with cache – but still insert missing entries.

def process(link: str)
   try:
       is_exist(link)
   except LookupError:
       # do something
       # put result to database
user_53499160

Typical use of lru_cache

I am not sure why you are not applying lru_cache as decorator:

@lru_cache(maxsize=None)
def is_exist(link :str) -> bool:
    if query_db(link) is True:
         return True
    return False

As long as the repeated calls to is_exist are going to use the same argument link the cache is going to be effective.

Please note maxsize=None which is going to ensure the limit of cache size is much larger than the default 128 (and restricted only by the RAM size).

Own implementation of the cache

lru_cache does not offer flexibility to cache the result depending on the return value of the function. However, you can easily implement it yourself using set:

cache = set()

def is_exist(link :str) -> bool:
    if query_db(link) is True:
         return True
    return False

def process(link: str)
   if link in cache:
      return True

   if is_exist(link) is True:
       cache.add(link)
       return
user_65739020

You need the test for the link and if it doesn't exist then create it in the same call - probably simplest to get rid of a cached is_exists and cache the process() function, but rename it to e.g. ensure_link:

@lru_cache
def ensure_link(link: str)
   if query_db(link) is True:
       return
   # do something
   # put result to database
   return

Either that or do the caching yourself in a function attribute or similar - it's not very difficult

user_23186490
Related