Config Drift Detector

Project roadmap

1. 01

   Design the detector

   ~2h

   Write DESIGN.md: the inventory (servers with groups), the baseline layout (baselines/GROUP/path/to/file, plus optional per-server overrides), which files are checked for each group, the normalization rules (remove comments and trailing spaces, collapse blank lines, and optionally sort lines for files where order doesn't matter), the report format, and the exit codes (0 no drift, 1 drift found and 2 errors). Done when: DESIGN.md defines inputs, rules and outputs.

2. 02

   Fetch baselines

   ~1.5h

   Get the approved files either from a local clone of the baseline repository (with git pull) or over HTTPS with curl from raw file URLs (with --fail, a timeout and a token from an environment variable), recording the commit or version used. Stop with a clear error when a baseline can't be fetched. Done when: the baselines for every group are available locally with their version recorded.

3. 03

   Collect live files over SSH

   ~2h

   For each server, fetch every file to check in a single SSH session (for example by running a small script remotely that prints each file with a separator, or with tar over ssh), using ssh -o BatchMode=yes -o ConnectTimeout=5 and read-only commands, and handle files that don't exist or can't be read (sudo -n only if configured). Done when: live files are collected for every reachable server, and unreachable ones are recorded.

4. 04

   Normalize and compare

   ~2h

   Apply the normalization rules with sed and awk to both the baseline and the live file, compare them with diff -u, and record each result: match, drift (with the diff), missing on the server, or extra files if checked. Done when: comparing a changed test file shows exactly the meaningful changes, and comment-only changes are ignored.

5. 05

   Check many servers in parallel

   ~1.5h

   Run servers in parallel with a limit (background jobs and wait -n, or xargs -P), each writing its results to its own folder, so outputs never mix. Continue past failures and include them in the report. Use a lock so two runs can't overlap. Done when: checking many servers is much faster than one at a time, and one bad server doesn't affect the others.

6. 06

   Report and track drift

   ~1.5h

   Use awk to build a summary table (server, files checked, matching, drifted, missing and errors), then list each drifted file with its diff. Save each run's results with a timestamp, and add a --since-last option showing only drift that's new since the previous run. Optionally send the summary to a webhook with curl. Done when: the report makes drift easy to see, and new drift is highlighted.

7. 07

   Test it

   ~2h

   Test with a few SSH servers in Docker containers with configuration files you change on purpose (a real change, a comment-only change, a missing file, and an unreachable server). Write bats-core tests for the normalization and comparison functions, and run shellcheck. Done when: the tests pass, and shellcheck reports no warnings.

Resources

• Docsssh(1) man page
• DocsGNU diffutils manual
• DocsGNU Awk user's guide
• Docscurl man page
• Docsbats-core documentation