Logging from multi processes corrupt file?

user_4935567 0
Viewed 184

I use Django served through gunicorn with four gevent workers and have some logging configuration, specially it uses RotatingFileHandler. The python documentation states that:

Although logging is thread-safe, and logging to a single file from multiple threads in a single process is supported, logging to a single file from multiple processes is not supported, because there is no standard way to serialize access to a single file across multiple processes in Python.

Therefore in my current config there's only one parent process for gunicorn and has some threads so the logging should not corrupt the file, but in my production I got backup files corrupted but could not reproduce the situation in my local setup.

Should I assume that the config is fine and continue using it or should I go for a different approach (e.g. multiprocessing.Queue or SocketHandler)?

gunicorn python-logging logging python
1 Answers

If you have four event workers, that means there are four distinct processes doing request handling and logging - so you should probably use SocketHandler as described here. I usually use Supervisor to run the listener process.

user_1189030
Related