After updating redis cluster I started getting max connections error from redis server, to fix this I set inactive connection timeout (https://redis.io/topics/clients here it stated with newer versions it now defaults to not have this timeout). This fixed the issue however after a few days celery could not make new connections because the sockets were clogged up with old connections in CLOSE_WAIT state. I've tried doing celery.close() and celery.backend.client.connection_pool.disconnect() at the end of these celery instance life cycle but both seem to have no effect on the amount of redis connections open.
I also tried refactoring my app to singleton celery approach - to do this I had to use thread lock on functions that called celery tasks synchronously send_task(..).get() to avoid concurrent redis pool calls, which is the reason it was made with dynamic celery instances in the first place, however with this approach I got wrong number of arguments for 'subscribe' command error occasionally only on production, that I couldnt find any info on.
Im using celery 4.4.7, python redis client 3.2.0 and redis engine version is 5.0.6 Any ideas appreciated.