Word Frequency Report
Build a Scala command-line tool that counts how often each word appears in a text, a file on your computer or a web page downloaded with the http4s client, and prints a ranked report: the most frequent words with their counts and percentages, skipping common words (stop words) from a list you can change. It's a small program written in the functional style used by many Scala teams: the program is a Cats Effect IOApp, reading files and the network are effects kept separate from the pure counting logic, which is easy to test. You'll learn Scala collections and groupBy, text processing, Cats Effect basics (IO and resources), the http4s client, and testing with munit.
- Estimate
- ~7.5h
- Steps
- 6
- Completed by
- 0
- Proposed by
- codeseed.app
sbt · Cats Effect · http4s
Project roadmap
- 01
Define the input and output
~1hWrite README.md: the command takes a file path or an http(s) URL, and options --top N (default 20), --stopwords FILE and --min-length; the output is a ranked table with the word, its count and its share of all counted words, plus the total and unique word counts. Create an sbt project with Cats Effect, the http4s Ember client and munit. Done when: README.md shows the options and example output.
- 02
Write the counting logic
~1.5hWrite pure functions: split text into words (lowercase, letters and apostrophes only, supporting accented letters), remove stop words and words shorter than the minimum, count with groupMapReduce or groupBy, and rank by count then alphabetically. Done when: unit tests on small texts give exactly the expected counts and order.
- 03
Read files and web pages
~2hIn an IOApp, read a file with fs2's Files API (or scala.io.Source inside IO.blocking), and for URLs, download the page with the http4s Ember client (as a Resource, with a timeout), removing HTML tags roughly before counting. Load the stop words file, or a built-in default list. Done when: the tool reads both a local file and a web page.
- 04
Handle errors
~1hHandle a missing file, a URL that fails or times out, a non-text response, an empty text, and invalid options, each with a clear message and a non-zero exit code (returning ExitCode.Error). Done when: every error case prints a helpful message and never a stack trace.
- 05
Print the report
~1hPrint the ranked table with aligned columns and percentages to one decimal, plus the totals. Add --csv to print the full list as CSV instead. Done when: the report is easy to read, and --csv output opens in a spreadsheet.
- 06
Test it
~1hWrite munit tests for the counting functions (punctuation, apostrophes, accents, stop words and ties), and munit-cats-effect tests for reading a temporary file and for the URL path with a fake http4s client. Done when: sbt test passes.
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack