CSV to JSON Converter Script

Project roadmap

1. 01

   Set up the script

   ~1h

   Create csv2json.pl with use strict and use warnings, and install Text::CSV (JSON::PP comes with Perl). Use Getopt::Long for options: an input file (or - for standard input), -o for an output file (default: standard output), --pretty and --help. Add samples/people.csv with a header and about 10 rows, including a field with a comma inside quotes. Done when: the script prints usage with --help and fails clearly for a missing file.

2. 02

   Read the CSV with Text::CSV

   ~1.5h

   Create a Text::CSV object with binary => 1 and auto_diag => 1, and read the header row with the header method. Then read each row with getline_hr, which returns a hash reference with the header names as keys. Handle quoted fields with commas and line breaks correctly. Done when: the script reads every row of the sample file, including the quoted field.

3. 03

   Convert values to the right types

   ~1.5h

   Write convert_value($text) that turns whole numbers and decimals (checked with a regular expression, so 007 and 1e5 are handled on purpose) into numbers, true and false (any upper or lower case) into JSON::PP::true and JSON::PP::false, empty strings into null (undef), and leaves everything else as text. Done when: the output has real numbers, booleans and nulls where they belong.

4. 04

   Write the JSON

   ~1h

   Collect the converted rows in an array and encode it with JSON::PP, using canonical => 1 so keys always appear in the same order, and pretty => 1 when --pretty is given. Encode the text as UTF-8 so accented characters survive. Write to the output file or standard output. Done when: the output is valid JSON (check it with a validator such as jq) and matches the input.

5. 05

   Handle broken input

   ~1h

   Report rows with more or fewer fields than the header as warnings with their line number, and skip them, or stop with an error when --strict is given. Also handle an empty file, a file with only a header, and duplicate column names. Print a summary to standard error with the number of rows converted and skipped. Done when: a CSV with a bad row reports it with its line number and still converts the rest.

6. 06

   Write tests

   ~1.5h

   Write tests in t/ with Test::More for convert_value (numbers, decimals, booleans, empty values, text that looks like a number) and for converting small CSV strings, including quoted commas and a bad row, to the expected data. Done when: prove -l t/ passes.

Resources

• DocsText::CSV on MetaCPAN
• DocsJSON::PP documentation
• DocsGetopt::Long documentation
• Docsperlretut: Perl regular expressions tutorial