JSON Lines Counter

Project roadmap

1. 01

   Set up the project and sample data

   ~1h

   Run go mod init and create main.go that reads the flags --field (required) and --top (default 10) with the flag package, plus a file path argument. Add sample.jsonl with at least 100 records containing a status field, a nested user.country field, and a few broken lines. Print usage and exit with code 2 when --field or the file is missing. Done when: running without arguments prints the usage.

2. 02

   Read the file line by line

   ~1.5h

   Open the file and read it with bufio.Scanner, raising its buffer size to 1 MB with scanner.Buffer so long lines don't fail. Decode each line into a map[string]any with json.Unmarshal. Count broken lines instead of stopping, and remember the first 5 of their line numbers. Skip empty lines. Done when: the program prints the number of valid and broken lines in sample.jsonl.

3. 03

   Count values of a field

   ~1.5h

   Write getField(record, path) that follows a dotted path such as user.country through nested maps, and returns the value and whether it was found. Count each value in a map[string]int, converting numbers and booleans to strings, and count records where the field is missing as (missing). Done when: counting status and user.country gives correct results for the sample file.

4. 04

   Print a sorted table

   ~1h

   Sort the counts from highest to lowest (ties in alphabetical order) with sort.Slice, and print the top N as an aligned table with the value, the count and the percentage, using text/tabwriter. Print the number of broken lines and their line numbers at the end. Done when: the output is sorted, aligned and shows correct percentages.

5. 05

   Read from a URL or stdin

   ~1.5h

   If the argument starts with http:// or https://, download it with net/http (with a 30-second timeout) and read the response body with the same scanner, without saving it to disk first. If the argument is -, read from standard input. Report HTTP errors, such as a 404, clearly. Done when: the same file gives the same counts whether it's read from disk, from a URL or piped into stdin.

6. 06

   Write tests

   ~1.5h

   Write table-driven tests for getField (top-level fields, nested fields, missing fields, non-object values) and for counting, using strings.NewReader as input so no files are needed. Add one test using httptest.NewServer to check the URL mode. Done when: go test ./... passes.

Resources

• DocsJSON Lines format
• Docsbufio package
• Docsencoding/json package
• DocsGo by Example