I have gunicorn running sync workers with a ThreadedConnectionPool as a singleton attribute using static methods in a class. I am running some background jobs in the master using the BackgroundScheduler of apscheduler. When a job runs on the master and a worker times out (due to a heavy database call), i start getting 500 errors on some subsequent requests on the new worker that call the DB with the error psycopg2.OperationalError: SSL error: decryption failed or bad record Mac
Here is my code.
class WriterDbConnectionPool:
__pool = None
@staticmethod
def get_db_connection_pool():
if not WriterDbConnectionPool.__pool:
logger.info("WriterDbConnectionPool initialising connection pool")
db_config = get_db_configs()
WriterDbConnectionPool.__pool = ThreadedConnectionPool(1, 10,
database=xxx,
user=xxx,
password=xxx,
host=xxx,
port=xxx
)
return WriterDbConnectionPool.__pool
@staticmethod
def close():
if WriterDbConnectionPool.__pool:
logger.info("WriterDbConnectionPool closing connection pool")
WriterDbConnectionPool.__pool.closeall()
WriterDbConnectionPool.__pool = None
I am running gunicorn with the following options
-k sync
--workers 3
--timeout 120
--bind 0.0.0.0:5000
--config gunicorn_config.py
--access-logfile -
--access-logformat
"%(p)s %(h)s %(l)s %(r)s %(s)s %(b)s %(f)s %(a)s %(L)s"
app:app
The code that invokes the DbConnectionPool call is below
@contextmanager
def get_db_connection():
try:
connection = DbConnectionPool.get_db_connection_pool().getconn()
yield connection
finally:
DbConnectionPool.get_db_connection_pool().putconn(connection)
How do I solve this issue? Should i use a different worker class?
My psycopg2 version is 2.7.5, flask is 0.12.2 and gunicorn 20.0.4