I'm aware this error has already been posted but the context of my issue is different. I have tried proposed solutions without success.
I'm using django-apscheduler to schedule some management commands, one of these uses ThreadPoolExecutor to speed up an otherwise lengthy process.
Unfortunately, it does not close connections, which eventually leads to:
exception:connection to server at "localhost" (::1), port 5432 failed: FATAL: sorry, too many clients already
I'm using PostgreSQL backend, and when I run the management command directly I can see in pgAdmin that the number of database sessions reduces after execution.
However, when ran by django-apscheduler they accumulate and eventually cause FATAL: sorry, too many clients already.
I've tried calling close_old_connections() from from django.db import close_old_connections but that doesn't seem to do anything.
Can someone point me in the right direction?
This is what my management command looks like:
import concurrent
from django.core.management.base import BaseCommand
from data_model.models import DataModel
class Command(BaseCommand):
help = "Do something interesting"
def handle(self, *args, **kwargs):
with concurrent.futures.ThreadPoolExecutor(10) as executor:
list(executor.map(DataModel.objects.do_something_interesting))
I did also try adding expiration time CONN_MAX_AGE to DB config but that didn't help:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.postgresql_psycopg2",
"NAME": env.str("POSTGRES_DB"),
"USER": env.str("POSTGRES_USER"),
"PASSWORD": env.str("POSTGRES_PASSWORD"),
"HOST": env.str("POSTGRES_HOST"),
"PORT": 5432,
"CONN_MAX_AGE": 60
},
}
Management command execution completed where the arrow is:
