Django persistent database connection

In Django trunk, edit django/db/__init__.py and comment out the line:

signals.request_finished.connect(close_connection)

This signal handler causes it to disconnect from the database after every request. I don't know what all of the side-effects of doing this will be, but it doesn't make any sense to start a new connection after every request; it destroys performance, as you've noticed.

I'm using this now, but I havn't done a full set of tests to see if anything breaks.

I don't know why everyone thinks this needs a new backend or a special connection pooler or other complex solutions. This seems very simple, though I don't doubt there are some obscure gotchas that made them do this in the first place--which should be dealt with more sensibly; 5ms overhead for every request is quite a lot for a high-performance service, as you've noticed. (It takes me 150ms--I havn't figured out why yet.)

Edit: another necessary change is in django/middleware/transaction.py; remove the two transaction.is_dirty() tests and always call commit() or rollback(). Otherwise, it won't commit a transaction if it only read from the database, which will leave locks open that should be closed.

Disclaimer: I have not tried this.

I believe you need to implement a custom database back end. There are a few examples on the web that shows how to implement a database back end with connection pooling.

Using a connection pool would probably be a good solution for you case, as the network connections are kept open when connections are returned to the pool.

• This post accomplishes this by patching Django (one of the comments points out that it is better to implement a custom back end outside of the core django code)
• This post is an implementation of a custom db back end

Both posts use MySQL - perhaps you are able to use similar techniques with Postgresql.

Edit:

• The Django Book mentions Postgresql connection pooling, using pgpool (tutorial).
• Someone posted a patch for the psycopg2 backend that implements connection pooling. I suggest creating a copy of the existing back end in your own project and patching that one.

This is a package for django connection pool: django-db-connection-pool

pip install django-db-connection-pool

You can provide additional options to pass to SQLAlchemy's pool creation, key's name is POOL_OPTIONS:

DATABASES = {
    'default': {
        ...
        'POOL_OPTIONS' : {
            'POOL_SIZE': 10,
            'MAX_OVERFLOW': 10
        }
        ...
     }
 }