Automated Log Rotation and Archival Tool

Project roadmap

1. 01

   Find rotation candidates

   ~1.5h

   Write a script using File::Find to walk the configured log folders, matching files by a pattern (such as *.log), and select those larger than a size limit or older than an age limit, using stat for the size and modification time. Skip files that are already compressed, and symbolic links. Print each candidate with the reason it was selected. Done when: on a test folder with files of different sizes and dates, exactly the right files are listed.

2. 02

   Compress and archive

   ~2h

   Compress each candidate with Compress::Zlib (gzopen and gzwrite, reading in chunks) into archive/YYYY/MM/DD/name.log.gz, creating the folders with File::Path. Write to a temporary name first, then verify the result by decompressing it and comparing its size (or a checksum) with the original. Only after that, rename the archive to its final name and truncate or delete the original. Done when: archived logs decompress to exactly the original content.

3. 03

   Prune old archives

   ~1h

   Delete archived files older than the retention period (for example 30 days), then remove archive folders that became empty. Never delete anything outside the archive folder: check every path before deleting it. Done when: only archives older than the retention period are deleted.

4. 04

   Add a config file and dry run

   ~1.5h

   Read settings from a config file (a simple key = value format, or YAML with YAML::Tiny): the log folders, patterns, size and age limits, archive folder and retention days. Validate them, and refuse to run when a folder doesn't exist. Add --dry-run, which prints every action it would take without changing anything, and --config to choose the file. Done when: --dry-run lists the planned actions, and the files are untouched afterwards.

5. 05

   Log actions and prepare for cron

   ~1.5h

   Log every action (compressed, verified, deleted and skipped, with sizes) and every error with a timestamp to a log file, and print a short summary at the end. Use a lock file (flock) so two runs never overlap. Exit with a non-zero code on errors. Add an example crontab line to README.md. Done when: running the tool from cron works, and a second overlapping run exits immediately.

6. 06

   Test it

   ~1.5h

   Write tests with Test::More that build a temporary folder with files of set sizes and dates (using utime to change the dates), then check selection, archiving and verification, pruning, dry run and the refusal to delete outside the archive. Done when: prove -l t/ passes.

Resources

• DocsFile::Find (Perl documentation)
• DocsCompress::Zlib on MetaCPAN
• DocsFile::Path (Perl documentation)
• DocsPerl stat function
• Docscrontab(5) man page