how to set connection timeout in flask redis cache

user_9920156 0
Viewed 1771

I'm trying to use redis cache with my python code, below code works fine and it sets the keys perfectly. I wanted to set timeout when its not able to connect to redis or if the ports are not open. unfortunately I could not able to find any document on how to pass the timeout to the connection parameters.

Following is my code.

from flask import Flask, render_template
from flask_caching import Cache

app = Flask(__name__, static_url_path='/static')

config = {
    "DEBUG": True,          
    "CACHE_TYPE": "redis",
    "CACHE_DEFAULT_TIMEOUT": 300,
    "CACHE_KEY_PREFIX": "inventory",
    "CACHE_REDIS_HOST": "localhost",
    "CACHE_REDIS_PORT": "6379",
    "CACHE_REDIS_URL": 'redis://localhost:6379'
}

cache = Cache(app, config=config)
socket_timeout = 5


@app.route('/')
@cache.memoize()
def dev():
  # some code
  return render_template("index.html", data=json_data, columns=columns)

when its not able to connect it waits for long time and throws the following error:

Traceback (most recent call last):
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/flask_caching/__init__.py", line 771, in decorated_function
    f, *args, **kwargs
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/flask_caching/__init__.py", line 565, in make_cache_key
    f, args=args, timeout=_timeout, forced_update=forced_update
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/flask_caching/__init__.py", line 524, in _memoize_version
    version_data_list = list(self.cache.get_many(*fetch_keys))
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/flask_caching/backends/rediscache.py", line 101, in get_many
    return [self.load_object(x) for x in self._read_clients.mget(keys)]
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/redis/client.py", line 1329, in mget
    return self.execute_command('MGET', *args, **options)
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/redis/client.py", line 772, in execute_command
    connection = pool.get_connection(command_name, **options)
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/redis/connection.py", line 994, in get_connection
    connection.connect()
  File "/Users/amjad/.virtualenvs/inventory/lib/python3.7/site-packages/redis/connection.py", line 497, in connect
    raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 60 connecting to localhost:6379. Operation timed out.

Thanks in advance.

redis-cluster flask redis python
2 Answers

This question is fairly old but came across this exact problem just now and I found a solution. Leaving here for posterity for future readers.

According to the documentation at https://flask-caching.readthedocs.io/en/latest/index.html, the CACHE_TYPE parameter:

Specifies which type of caching object to use. This is an import string that will be imported and instantiated. It is assumed that the import object is a function that will return a cache object that adheres to the cache API.

So make a modified version of their redis function, found in flask_caching.backends.cache like so:

def redis_with_timeout(app, config, args, kwargs):

    try:
        from redis import from_url as redis_from_url
    except ImportError:
        raise RuntimeError("no redis module found")

    # [... extra lines skipped for brevity ...]

    # kwargs set here are passed through to the underlying Redis client
    kwargs["socket_connect_timeout"] = 0.5
    kwargs["socket_timeout"] = 0.5

    return RedisCache(*args, **kwargs)

And use it instead of the default redis like so:

CACHE_TYPE = 'path.to.redis_with_timeout'

And the library will use that one instead, with the custom kwargs passed into the underlying Redis client. Hope that helps.

user_58596210

From latest document, there is an CACHE_OPTIONS config passed to almost every types of cache backends as keyword arguments:

Entries in CACHE_OPTIONS are passed to the redis client as **kwargs

We can simply pass additional settings like this:

from flask import Flask
from flask_caching import Cache

app = Flask(__name__)

config = {
    "CACHE_TYPE": "redis",
    ...
    "CACHE_REDIS_HOST": "localhost",
    "CACHE_REDIS_PORT": "6379",
    "CACHE_REDIS_URL": 'redis://localhost:6379',
    "CACHE_OPTIONS": {
        "socket_connect_timeout": 5,    # connection timeout in seconds
        "socket_timeout": 5,            # send/recv timeout in seconds
    }
}

cache = Cache(app, config=config)
user_18254430
Related