Incremental Backup Verifier

Project roadmap

1. 01

   Set up the scripts and config

   ~1.5h

   Create bin/manifest.sh, bin/diff.sh and bin/verify.sh, each starting with set -euo pipefail and a shared lib.sh with logging helpers (timestamped info, warn and error lines) and a die function. Read settings (source directory, backup directory, remote host, alert URL) from a config file whose path is given with -c. Validate that required settings are present. Done when: each script prints usage and fails clearly on a missing config.

2. 02

   Create a manifest

   ~2h

   manifest.sh walks the source directory with find -type f -print0 and, for every file, records the relative path, size and SHA-256 (sha256sum), saving the result sorted by path to manifests/<date>.tsv. Handle file names with spaces and newlines safely by using null-separated lists. Done when: a manifest lists every file of a test directory, and running it twice on unchanged files gives identical manifests.

3. 03

   Compare two manifests

   ~2h

   diff.sh takes two manifests and uses awk to classify each path as added, deleted, modified (same path, different hash) or unchanged, and prints counts plus the list of changed paths. Also report files whose size changed by more than 50%, which may be a sign of corruption. Done when: after adding, editing and deleting files in a test directory, diff.sh reports exactly those changes.

4. 04

   Verify a backup archive

   ~2.5h

   verify.sh extracts a backup archive (tar.gz) into a temporary directory created with mktemp -d, computes a manifest of the extracted files, and compares it with the manifest recorded at backup time. Any missing, extra or mismatched file makes the check fail. A trap removes the temporary directory on exit, even after an error. Done when: an intact archive passes, and an archive with one corrupted file fails and names that file.

5. 05

   Verify remote backups over SSH

   ~2h

   Add a remote mode: when the backup lives on another server, run sha256sum there over SSH (with BatchMode=yes and a timeout) to compute hashes on the remote side, instead of copying the whole backup. Compare the result with the local manifest. Document the SSH key setup in the README. Done when: verifying a backup on a second machine or a local VM works over SSH.

6. 06

   Prevent overlaps and report results

   ~2h

   Use flock on a lock file so two verifications never run at the same time. At the end, if an alert URL is configured, report success or failure with curl (--fail, a 10-second timeout, retries), for example to a healthchecks.io-style ping URL. Use distinct exit codes: 0 verified, 1 verification failed, 2 configuration or runtime error. Done when: a second run started during the first exits immediately, and the result is reported to a test URL.

7. 07

   Test the scripts

   ~2h

   Write tests with bats-core that build test directories and archives: an unchanged backup, added and deleted files, a corrupted file, a file name with spaces, and a missing archive. Check the output and exit codes. Also run shellcheck on every script and fix all warnings. Done when: bats tests/ passes and shellcheck reports no issues.

Resources

• DocsGNU Bash manual
• DocsGNU Awk user's guide
• Docsflock(1) manual page
• Docsbats-core documentation
• DocsShellCheck