Mailbox Attachment Extractor

Project roadmap

1. 01

   Define the rules and outputs

   ~1.5h

   Write README.md: the input (an mbox file or a Maildir folder), the rules (sender pattern, subject pattern, date range, file types and maximum size), the output folder layout (for example attachments/SENDER/YYYY-MM/), the file naming rules (clean names, no paths, no hidden files, and a number added for duplicates), and the database tables (messages and attachments with checksums). Set up a cpanfile with Email::MIME, Mail::Box (or Email::Folder), DBI and DBD::SQLite. Done when: README.md defines the rules, layout and tables.

2. 02

   Read the messages

   ~2h

   Open the mailbox with Mail::Box::Manager (or read the mbox or Maildir yourself) and parse each message with Email::MIME, extracting the Message-ID, sender, subject and date (decoding encoded headers). Record parsing errors per message and continue. Done when: the tool lists every message in a sample mailbox with its details.

3. 03

   Find matching attachments

   ~2h

   Walk each message's MIME parts recursively (with walk_parts), and pick out attachments by their disposition and file name, decoding encoded file names. Apply the rules (sender, subject, date, type by extension and content type, and size). Done when: the matching attachments of a sample mailbox are listed correctly, including ones inside nested parts.

4. 04

   Save attachments safely

   ~2h

   Clean each file name (keep only the base name, replace unsafe characters, block names starting with a dot and limit the length), create the target folders, and write the decoded content to a temporary file, then rename it, adding a number when the name exists. Compute a SHA-256 checksum and skip saving files already saved with the same checksum. Never execute or open the files. Done when: attachments are saved with safe, unique names, and duplicates are skipped.

5. 05

   Record metadata with DBI

   ~1.5h

   Store each processed message (Message-ID, sender, subject and date) and its saved attachments (file name, path, size and checksum) in SQLite within one transaction per message, with a unique index on the Message-ID so running the tool again skips messages already processed. Add a --report option that summarizes the database (attachments per sender and month). Done when: re-running on the same mailbox processes nothing new, and the report is correct.

6. 06

   Test it

   ~1.5h

   Write Test::More tests with small sample mailboxes created in the test (built with Email::MIME): nested attachments, encoded file names, dangerous names such as ../../etc/passwd, rules matching and not matching, duplicates, a broken message, and re-running. Done when: prove -l t/ passes.

Resources

• DocsEmail::MIME on MetaCPAN
• DocsMail::Box on MetaCPAN
• DocsDBI on MetaCPAN
• ArticleMIME (Wikipedia)
• DocsOWASP: file upload cheat sheet (safe file names)