Realtime Collaboration Server

Project roadmap

1. 01

   Design the protocol

   ~2.5h

   Write DESIGN.md: documents are plain text; a change is an operation (insert text at a position, or delete a range) with the document version it was based on and a client id. The server accepts a change only if its base version is current, assigns the next version and broadcasts it; otherwise it answers with the missed changes so the client can adjust and resend. Describe the messages (join, change, ack, reject and presence), snapshots every N changes, and recovery. Done when: DESIGN.md explains the protocol with an example of two conflicting edits.

2. 02

   Build the WebSocket server

   ~2.5h

   Create a Composer project with Ratchet (or ReactPHP with a WebSocket library). Accept connections, parse JSON messages with validation, let clients join a document, and keep track of connections per document. Write a simple browser client with a textarea that sends and applies changes. Done when: two browsers joined to the same document are connected and receive each other's presence.

3. 03

   Order and apply changes

   ~3h

   Keep each open document's text and version in memory. Accept changes whose base version matches, apply them, increase the version, acknowledge the sender and broadcast to others; reject stale changes with the missing changes so the client can transform its edit (adjusting positions) and resend. Ignore repeated change ids. Done when: two people typing in the same document at the same time end up with identical text.

4. 04

   Persist changes and snapshots

   ~2.5h

   Store every accepted change in PostgreSQL (document, version, operation and author) and a snapshot of the full text every 100 changes. Because database calls block, run them through an async PostgreSQL client for ReactPHP or in a worker process, so the event loop never stops. Done when: changes and snapshots are saved without slowing down live editing.

5. 05

   Recover after a restart

   ~2h

   When a document is opened and isn't in memory, load its latest snapshot and apply the changes after it. After a server restart, clients reconnect (with increasing waits in the browser client) and resync from their last known version. Unload documents nobody has used for a while. Done when: restarting the server mid-session loses no accepted change, and clients continue editing.

6. 06

   Add limits and monitoring

   ~1.5h

   Limit message size, document size, changes per second per connection, and connections per document. Log connections, rejections and errors, and expose a small status endpoint with open documents, connections and changes per minute. Done when: exceeding a limit gets a clear rejection, and the status endpoint shows live numbers.

7. 07

   Test it

   ~2.5h

   Write PHPUnit tests for applying and transforming operations (as pure functions), and integration tests that start the server and connect several WebSocket clients sending random concurrent changes, checking they all end with the same text and that it matches the database after recovery. Done when: all tests pass.

Resources

• DocsRatchet: WebSockets for PHP
• DocsReactPHP documentation
• ArticleOperational transformation (Wikipedia)
• DocsMDN: the WebSocket API
• DocsComposer: getting started