Graceful caching with Python and Redis

Viewed 4111

I want to cache data from external service in graceful manner: the old data, though expired, is being served until the worker successfully fetches new data. The data is not time-critical, but the lack of data (external service down) would prevent the service running, thus using a persistent cache.

Currently

  • I store fetch timestamp in a separate Redis key

  • I cache the data indefinitely until the worker fetches new one (I do not set expiration time)

Questions

  • Is this correct way of doing graceful caching with Redis?

  • Can I natively get key updated timestamp from Redis, so I do not need to store this information myself

This is the code:

def set_data(self, data):
    self.redis.set("bitcoinaverage", pickle.dumps(data))
    self.redis.set("bitcoinaverage_updated", calendar.timegm(datetime.datetime.utcnow().utctimetuple()))

def get_data(self):
    return pickle.loads(self.redis.get("bitcoinaverage"))

def is_up_to_date(self):

    last_updated = datetime.utcfromtimestamp(self.redis.get("bitcoinaverage_updated"))

    if not last_updated:
        return False

    return datetime.datetime.utcnow() < last_updated + self.refresh_delay

def tick(self):
    """ Run a periodical worker task to see if we need to update.
    """
    if not self.is_up_to_date():
        self.update()
2 Answers
Related