CSV Index Builder
Build a C tool that makes looking things up in a huge CSV file fast: it scans the file once, builds an index for a chosen column (a hash table mapping each value to the positions of its rows in the file), and then answers lookups instantly by jumping straight to the matching rows, instead of reading the whole file every time. The index can be saved to a file and reused until the CSV changes. It handles quoted fields correctly, works on files larger than memory, and reports duplicates and malformed rows. You'll learn parsing CSV properly in C, hash tables with collision handling, file offsets with fseeko, binary file formats, memory management, and measuring performance.
- Estimate
- ~12h
- Steps
- 6
- Completed by
- 0
- Proposed by
- codeseed.app
Make · POSIX · CMake
Project roadmap
- 01
Define the tool
~1.5hWrite README.md: the commands (build CSV --column NAME, which creates CSV.idx, and lookup CSV VALUE, which prints matching rows), how quoted fields are handled (RFC 4180), what happens with duplicate keys (all matching rows are returned), and the index file format (a header with the CSV's size, modification time and column, then the entries). Set up the build with Make or CMake using -Wall -Wextra -O2. Done when: README.md defines the commands and the index format.
- 02
Parse CSV records
~2.5hWrite a parser that reads the file with a buffer and returns records one at a time with the byte offset where each starts, handling quoted fields with commas, doubled quotes and line breaks inside quotes. Find the chosen column by its name in the header. Report malformed rows (an unclosed quote or the wrong number of fields) with their line number, and continue. Done when: tests with tricky quoted records give correct fields and offsets.
- 03
Build the hash index
~2.5hImplement a hash table (open addressing with linear probing, or chaining) with a good string hash such as FNV-1a, mapping each key to a list of row offsets and growing when it's too full. Scan the CSV once and insert every row's key with its offset. Report the number of rows, unique keys and duplicates. Done when: building the index for a file with a million rows works, and every key maps to the right offsets.
- 04
Look up rows
~1.5hFor a lookup, find the key in the hash table and, for each offset, jump to it with fseeko and parse just that record, printing it. Compare the time with a full scan of the file for the same value, and print both with --timing. Done when: lookups return the right rows and are much faster than a full scan.
- 05
Save and load the index
~2hWrite the index to a binary file (the header, then keys and offsets with fixed-size integers and a checksum), written to a temporary name and renamed. When looking up, load the saved index if the CSV's size and modification time still match the header, and otherwise rebuild it with a message. Handle a damaged index file by rebuilding. Done when: repeated lookups reuse the saved index, and changing the CSV triggers a rebuild.
- 06
Test and measure
~2hWrite tests for the parser, the hash table (including many collisions) and index saving and loading, and a script comparing lookup results with a simple grep or awk-based search on generated files. Generate a large CSV (for example 5 million rows), record build and lookup times and memory use in README.md, and run everything under valgrind or AddressSanitizer. Done when: all tests pass with no memory errors, and README.md shows the measurements.
Resources
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack