One of the less interesting parts of running your own server is dealing with logs.
They’re useful when something goes wrong, but most of the time you don’t need to keep months of access logs, error logs and application output.
If they’re left alone, they can gradually consume a surprising amount of disk space.
I use EasyEngine to run WordPress in Docker containers, which means the logs aren’t necessarily in the usual locations you’d expect from a traditional Nginx or PHP installation.
I wanted to keep enough logs for debugging without letting them grow indefinitely.
Finding the logs
EasyEngine stores much of its persistent data in Docker volumes, including the logs generated by Nginx and PHP.
To see which volumes are taking up the most space, I use:
1cd /var/lib/docker/volumes2du -sh * | sort -hr
This gives a list of directories sorted by size, making it easier to identify anything that’s growing unexpectedly.
It’s a useful starting point, although I’d check the actual files before deleting anything.
Logs written inside Docker volumes are also separate from Docker’s own container logging system. Depending on how a container is configured, you might need to manage both.
Setting up logrotate
Linux has a utility called logrotate which handles rotating, compressing and removing old log files.
Rather than manually deleting logs, I can configure how long to keep them.
For example, an EasyEngine application log can be configured like this:
1/opt/easyengine/logs/ee.log { 2 daily 3 rotate 14 4 compress 5 missingok 6 notifempty 7 create 0640 root root 8 delaycompress 9 dateext10}
This rotates the log daily and keeps fourteen rotated files.
Older logs are compressed to save space, while notifempty prevents empty files from being rotated.
delaycompress waits until the following rotation before compressing the most recently rotated log.
The resulting log files have dates in their filenames, making it easier to find the logs from a particular day.
Rotating logs used by containers
Rotating an application log is straightforward when the application opens and closes the file for each write.
It’s more complicated when a process keeps the log file open.
Renaming a log file doesn’t necessarily make Nginx or PHP start writing to the new one. The process might continue writing to the old file, even though it has been rotated.
For Nginx, the usual solution is to signal the process to reopen its logs.
A simplified configuration might look like this:
1/var/log/nginx/*.log { 2 daily 3 rotate 14 4 compress 5 missingok 6 notifempty 7 create 0640 www-data adm 8 sharedscripts 9 postrotate10 /usr/sbin/nginx -s reopen11 endscript12}
The important part is postrotate.
Once the log files have been rotated, Nginx is told to reopen them so that it starts writing to the new files.
With Docker, this command needs to run in the appropriate container rather than assuming Nginx is running directly on the host.
The same principle applies to other processes, although the signal or command needed to reopen their logs may be different.
Testing the configuration
I don’t like making changes to server maintenance tasks without checking what they’re going to do.
Logrotate has a debug mode which shows how it would process the configuration without actually rotating the files.
1sudo logrotate -d /etc/logrotate.conf
You can also test a specific configuration file:
1sudo logrotate -d /etc/logrotate.d/easyengine
This is particularly useful when matching logs across multiple Docker volumes.
Once the configuration is correct, I leave logrotate to handle the housekeeping.
Keeping enough logs
The temptation is to delete logs as soon as disk space becomes a problem, but that can leave you without the information you need to investigate an incident.
Fourteen days is a reasonable starting point for some application logs, but retention depends on how frequently the website is used and what the logs are needed for.
It’s also worth monitoring disk usage rather than relying entirely on rotation to prevent storage problems.
The end result isn’t particularly exciting. Logs get rotated, old files get compressed, and the server doesn’t gradually fill up with information I’m unlikely to need.
Which is exactly what I wanted.