Schema Evolution Tool

Project roadmap

1. 01

   Define the compatibility rules

   ~2.5h

   Write RULES.md: which changes break clients that read the data (such as adding a new required field they don't expect, or widening a type) and which break clients that send data (such as a field becoming required, a type narrowing, an enum losing values, or a maximum decreasing), and which are safe. Decide which subset of JSON Schema you support (type, properties, required, enum, items, minimum, maximum, maxLength and $ref). Done when: RULES.md lists every change with its classification and an example.

2. 02

   Load and resolve schemas

   ~2.5h

   Create an sbt project with Circe, Cats Effect and the http4s Ember client. Load a schema from a file path or a URL (with a timeout and clear errors), parse it with Circe into your own schema model, and resolve local $ref references (#/$defs/...), detecting cycles. Report unsupported keywords as warnings. Done when: sample schemas with references load into the model.

3. 03

   Compare schemas

   ~3h

   Write a recursive comparison of the old and new schema models that walks properties and array items together, producing a list of changes, each with a JSON path (such as $.order.items[].price), a kind, the old and new value, and a classification from your rules. Keep the comparison as pure functions. Done when: comparing sample schema pairs finds exactly the expected changes.

4. 04

   Report the results

   ~1.5h

   Print a report grouped by breaking and safe changes with their paths and explanations, and a summary. Add --format json for machines, a --mode option to check compatibility for readers, writers or both, and exit codes: 0 when no breaking changes, 1 when there are some, and 2 on errors. Done when: the report is clear, and the exit codes are right.

5. 05

   Compare many versions

   ~1.5h

   Support comparing a folder of versioned schemas (v1.json, v2.json and so on) in order, reporting each step, and checking a new schema against all earlier ones if requested. Load several schemas concurrently with Cats Effect's parTraverse, with a limit. Done when: comparing a folder of versions reports each step's changes.

6. 06

   Handle edge cases

   ~1.5h

   Handle schemas with cycles through $ref, unsupported keywords (reported, not silently ignored), invalid JSON (with line and column), unreachable URLs, and schemas that are identical. Done when: each case gives a clear message and the right exit code.

7. 07

   Test it

   ~2h

   Write munit tests for every rule in RULES.md with small schema pairs, and property-based tests with ScalaCheck: comparing a schema with itself finds no changes, and adding an optional property is never breaking. Test URL loading with an in-memory http4s server. Done when: sbt test passes.

Resources

• DocsJSON Schema: getting started
• DocsUnderstanding JSON Schema
• ArticleConfluent: schema evolution and compatibility
• Docshttp4s client documentation
• DocsScalaCheck user guide