CSV Query CLI

Project roadmap

1. 01

   Define the command

   ~1.5h

   Write README.md with the command (csvq FILE --select name,price --where 'price > 100 and country = "RO"' --sort price --desc --limit 10 --count --format table|csv|json), the supported operators (=, !=, <, <=, >, >=, contains, and, or, not and parentheses), how types are treated (numbers compared as numbers when both sides parse as numbers, otherwise as text), and examples. Create the project with csv, serde, serde_json, clap and thiserror. Done when: README.md defines the command and filter language.

2. 02

   Stream records and select columns

   ~1.5h

   Read the file with csv::Reader (from a path, or standard input with -), look up the selected columns by header name (with a clear error listing the available columns when one doesn't exist), and write the selected fields of each record to standard output with csv::Writer, record by record. Done when: selecting columns from a large file works with low, steady memory use.

3. 03

   Parse filter expressions

   ~2.5h

   Write a tokenizer and a recursive descent parser for the filter language that builds an expression tree (an enum), with and binding tighter than or, and errors that point to the position of the problem (such as expected a value after > at position 9). Check column names in the expression against the header. Done when: unit tests parse valid expressions correctly and give clear errors for invalid ones.

4. 04

   Filter, count, sort and limit

   ~2h

   Evaluate the expression for each record, comparing as numbers or text as the rules say. With --count, only print the number of matches. With --limit and no sort, stop reading as soon as enough rows match. Sorting needs the matching rows in memory, so keep only the top N rows in a binary heap when --limit is given, and warn when sorting a very large result without a limit. Done when: filters, counts, sorting and limits give the same results as the same query in another tool.

5. 05

   Output formats and errors

   ~1.5h

   Print results as an aligned table (for small results), CSV, or JSON lines (with serde_json). Report malformed CSV records with their line number, skipping them with --skip-errors or stopping otherwise. Define the errors with thiserror and show clear messages without panics. Done when: every format works, and bad input gives clear messages.

6. 06

   Test and measure

   ~1.5h

   Write unit tests for the parser and evaluator, and integration tests running the binary on fixture files (with assert_cmd) for each option. Generate a multi-gigabyte CSV and record the time and peak memory for a few queries in README.md. Run cargo clippy. Done when: cargo test passes, clippy is clean, and README.md shows the measurements.

Resources

• Docscsv crate tutorial
• Docsclap derive tutorial
• Docsthiserror crate
• DocsCrafting Interpreters: parsing expressions (free book)
• Docsstd::collections::BinaryHeap