Async Reverse Proxy

Project roadmap

1. 01

   Design the proxy

   ~2h

   Write DESIGN.md: the TOML config (listen address, routes with a host, path prefix, backend URL, timeout and whether to strip the prefix, plus global limits), how a request is matched (host first, then the longest path prefix), which headers are added (X-Forwarded-For, X-Forwarded-Proto and a request id) and removed (hop-by-hop headers), the limits and their responses (413, 429, 503 and 504), the log line fields, and how reloading works. Done when: DESIGN.md covers routing, limits, logs and reloading.

2. 02

   Load the config and route requests

   ~2.5h

   Create a project with tokio, hyper (with hyper-util and http-body-util), serde, toml and tracing. Load and validate the config with Serde (valid URLs, no duplicate routes and positive timeouts), and build a router that matches a request to a route or returns 404. Forward matched requests to their backend with a pooled hyper client, streaming bodies both ways, rewriting the path when the prefix is stripped, and fixing up headers. Done when: requests reach the right backend according to host and path.

3. 03

   Add timeouts and limits

   ~2h

   Wrap the connection to the backend and the whole request in tokio::time::timeout (returning 504 when it passes), limit request bodies by Content-Length and while streaming (413), cap concurrent connections with a Semaphore (503 when full), and set a timeout for reading client headers. Done when: slow backends get 504, oversized bodies get 413, and too many connections get 503.

4. 04

   Rate limit per client

   ~1.5h

   Add a token bucket per client IP stored in a DashMap (or a Mutex-protected HashMap), with limits from the config, returning 429 with Retry-After when exhausted, and a background task removing idle entries so memory doesn't grow. Done when: a client sending too fast gets 429, and memory stays flat with many clients.

5. 05

   Log requests as JSON

   ~1h

   Use tracing with a JSON formatter to write one line per request: time, request id, client IP, method, host, path, route, backend, status, bytes in and out, and duration, plus errors with their cause. Pass the request id to backends in a header. Done when: every request produces a complete JSON log line.

6. 06

   Reload the config

   ~1.5h

   On SIGHUP (or when the file changes), load and validate the new config, and if it's valid, swap it in atomically (with arc-swap or an RwLock around an Arc), so new requests use it while requests in progress finish with the old one; if it's invalid, keep the old config and log the error. Shut down gracefully on Ctrl+C. Done when: changing a route and sending SIGHUP takes effect without dropping requests.

7. 07

   Test it

   ~2.5h

   Write integration tests that start the proxy and test backends on random ports: routing by host and path, prefix stripping, headers, each timeout, body limits, the connection limit, rate limiting, and reloading. Load test on your own machine with oha or wrk and record the added latency in README.md. Done when: cargo test passes, and README.md shows the overhead.

Resources

• DocsTokio tutorial
• Docshyper guides
• Docstracing crate documentation
• Docsarc-swap crate
• DocsMDN: X-Forwarded-For