Non-blocking Chat Server with epoll

Project roadmap

1. 01

   Open a non-blocking listening socket

   ~1.5h

   Create a CMake project using C++17 or newer. Create a TCP socket, set SO_REUSEADDR, bind it to a port from the command line (default 9000), listen, and make it non-blocking with fcntl and O_NONBLOCK. Create an epoll instance with epoll_create1 and register the listening socket for EPOLLIN. Wrap file descriptors in a small RAII class so they're always closed. Done when: the server starts and nc localhost 9000 connects.

2. 02

   Build the event loop

   ~2.5h

   Write the main loop around epoll_wait. When the listening socket is readable, accept every pending connection until accept returns EAGAIN, make each one non-blocking and register it. When a client socket is readable, read until EAGAIN. Store each client in a map from file descriptor to a Client struct. Log connections and disconnections. Done when: several nc clients can connect at once and the server logs what they send.

3. 03

   Handle partial reads and writes

   ~3h

   Give each client an input buffer and an output buffer. Append incoming bytes to the input buffer and extract complete messages ending in \n, keeping any incomplete remainder for later. To send, append to the output buffer and write as much as the socket accepts; if data remains, enable EPOLLOUT for that client and continue when it becomes writable, then disable EPOLLOUT once the buffer is empty. Done when: messages split across packets, or sent together, are handled correctly.

4. 04

   Add nicknames and broadcasting

   ~2h

   The first line from a client is its nickname: reject empty ones, names over 20 characters and names already in use. After that, every message is broadcast to all other clients as [nick] message, and joins and leaves are announced. Support two commands: /who lists connected users and /quit disconnects. Limit messages to 1 KB and cut off anything longer. Done when: three nc clients can chat with each other.

5. 05

   Handle disconnects and slow clients

   ~2h

   Remove clients cleanly when read returns 0, when EPOLLHUP or EPOLLERR is reported, or when a write fails with EPIPE (ignore SIGPIPE, or use MSG_NOSIGNAL). If a client's output buffer grows beyond 64 KB because it isn't reading, disconnect it so it can't use unlimited memory. Always remove the socket from epoll before closing it. Done when: killing a client abruptly, or sending to a client that never reads, doesn't crash or slow down the server.

6. 06

   Load test the server

   ~2.5h

   Write a load-test client (in C++ or Python with asyncio) that opens N connections (for example 2,000), gives each a nickname, sends messages at a steady rate and measures how long broadcasts take to arrive. Raise the open-file limit with ulimit -n if needed. Record the results, CPU and memory use in LOADTEST.md. Done when: the server handles at least 1,000 simultaneous clients and LOADTEST.md documents the results.

7. 07

   Test the message framing

   ~1.5h

   Move the logic that splits messages out of the input buffer into a function you can test without sockets, and write unit tests (GoogleTest or Catch2) for: one full message, a message split in three pieces, two messages in one read, an empty line, and a message over the size limit. Done when: ctest passes.

Resources

• Docsepoll(7) manual page
• DocsBeej's Guide to Network Programming
• Docssocket(7) manual page
• ArticleThe C10K problem
• DocsGoogleTest primer