I am using Django logger for logging along with gunicorn workers. Currently, I am testing with 3 workers and I am using the TimedRotatingFileHandler. I configured when = m and interval = 1, I tried sending some 1000 requests and it is writing in the new file and overwriting the old file as well.
Here is my config from settings.py
'debug': { 'level': 'DEBUG', 'filename': BASE_DIR + '/Log/debug.log', 'class': 'logging.handlers.TimedRotatingFileHandler', 'when': 'M', 'interval': 1, 'formatter': 'verbose' },
This will happen only if i use more than 1 gunicorn worker. But if I use 1 gunicorn worker this is not happening.
Advertisement
Answer
You shouldn’t be using file-based handlers directly from multiple processes concurrently, which is what you get if you use gunicorn
with this configuration. Instead, use a SocketHandler
or QueueHandler
to send logs from the webapp to a separate process which logs them to disk. The Python logging cookbook has more information and example code, which is too long to reproduce here.
Writing to a single file from multiple concurrent processes without them treading on each other isn’t supported in Python (and a lot of other languages too).