Web Server Log Analyzer
Build a Bash script that analyzes web server access logs (Apache or Nginx) using the classic Unix text tools, awk, grep and sort, and prints a clear report: the most requested pages, how many requests got each status code, the busiest client IPs (flagging any that send an unusual number of requests per minute), and a chart of traffic per hour drawn with text bars. It reads plain and gzip-compressed rotated logs transparently, and handles very large files quickly because awk processes everything in one pass. You'll learn awk programming (fields, patterns, arrays and printf), regular expressions, pipelines with sort and uniq, handling compressed files, and writing robust shell scripts.
- Estimate
- ~7.5h
- Steps
- 6
- Completed by
- 0
- Proposed by
- codeseed.app
awk · grep · sort
Project roadmap
- 01
Parse the log format
~1.5hWrite loganalyze.sh with set -euo pipefail, taking one or more log files. Write an awk program that extracts the IP, timestamp, method, path (without the query string) and status from the common and combined formats (splitting on spaces and quotes carefully, since user agents contain spaces), and skips and counts lines that don't match. Done when: the parsed fields for sample lines are printed correctly.
- 02
Count paths and status codes
~1hIn the same awk pass, count requests per path and per status code in awk arrays, then print the top N paths (a -n option) and every status code with its share, sorted with sort -rn (or awk's asort). Group status codes by class (2xx, 3xx, 4xx and 5xx) too. Done when: the counts match results from grep and wc on a sample log.
- 03
Find the busiest clients and anomalies
~1.5hRank client IPs by request count, and count requests per IP per minute (keyed by IP and the timestamp truncated to the minute) to flag IPs that exceed a threshold (a -t option) in any minute, showing their peak rate and when it happened. Done when: an IP added to a test log with a burst of requests is flagged.
- 04
Draw an hourly traffic chart
~1hCount requests per hour of the day, and print one line per hour with the count and a bar of # characters scaled so the busiest hour fills the width (a -w option). Done when: the chart shows the traffic pattern of a sample log clearly.
- 05
Read compressed rotated logs
~1hAccept any mix of plain and .gz files (such as access.log and access.log.2.gz), reading compressed ones with zcat (or gzip -dc), and a glob of rotated logs processed in order. Read from standard input when no file is given. Report missing files and continue. Done when: a set of rotated plain and compressed logs is analyzed as one.
- 06
Test it
~1.5hCreate small sample logs with known counts (including malformed lines, query strings and a burst from one IP), and write bats-core tests checking each section of the report. Run shellcheck on the script. Done when: the tests pass, and shellcheck reports no warnings.
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack