Django multiprocessing database concurrency

user_5945480 0
Viewed 641

I am stuck with the following database concurrency problem.

I am building a django webapp using psycopg2 to comunicate with a PostgreSQL database. The queries are run from different processes.

In order to lock rows all the queries are inside a transaction atomic block.

   with transaction.atomic():
       self.analysis_obj = Analysis.objects.select_for_update().get(pk=self.analysis_id)

However sometimes I get random error like:

  • 'no results to fetch',
  • 'pop from empty list',
  • 'error with status PGRES_TUPLES_OK and no message from the libpq'.

Any idea to deal with this problem?

Many thanks.

postgresql multiprocessing django python
1 Answers

For whom is interested I found a solution.

When using multiprocessing in Python, you should close all connection every time a process is spawned.

from django.db import connection
connection.close()
user_59454800
Related