Distributed Rate Service

Project roadmap

1. 01

   Design the service

   ~2h

   Write DESIGN.md: token bucket parameters (capacity and refill rate) per policy, keys per customer and policy, how the Redis script refills based on elapsed time and takes tokens, what the API returns (allowed, remaining tokens, and when to retry), how time is handled (use Redis's TIME, not each server's clock), and what happens when Redis fails (fail open, fail closed, or a local fallback limiter). Done when: DESIGN.md covers the algorithm, storage, API and failure modes.

2. 02

   Write the token bucket

   ~2h

   First implement the token bucket as a pure Scala function (state, now and cost to new state and decision) and test it thoroughly: refilling, bursts up to capacity, never going over capacity, and costs larger than one. Done when: property-based tests (with ScalaCheck) confirm the bucket never allows more than capacity plus refill over any period.

3. 03

   Store buckets in Redis atomically

   ~2.5h

   Write a Lua script that reads a bucket (tokens and last refill time) from a Redis hash, refills it using the current time from TIME, takes tokens if enough are available, saves it with an expiry, and returns the decision. Call it from Scala with redis4cats using EVALSHA, reloading it when Redis answers NOSCRIPT. Done when: the Redis version gives the same decisions as the pure function in tests.

4. 04

   Build the HTTP API

   ~2h

   Build an http4s service with POST /check (customer, policy and cost) returning the decision, remaining tokens and retry time, and GET /policies. Load policies from configuration, validated at startup. Add rate limit headers for callers to pass on. Done when: the API returns correct decisions, and invalid requests get clear errors.

5. 05

   Handle Redis failures

   ~2h

   Put a short timeout on every Redis call. When Redis is slow or unreachable, switch to a local in-memory token bucket per instance (with the limit divided by the expected number of instances), log it, and switch back when Redis recovers, using a simple circuit breaker. Make the behavior configurable per policy. Done when: stopping Redis keeps the service answering with the fallback, and restarting Redis restores shared limits.

6. 06

   Test across instances

   ~2.5h

   Write tests with Redis in Testcontainers: many fibers from two service instances checking the same customer at the same time never exceed the limit, refills over time (with a controllable delay), and the fallback during a Redis outage. Load test on your own machine and record decisions per second and latency in README.md. Done when: the tests pass, and README.md shows the load test results.

Resources

• ArticleToken bucket (Wikipedia)
• DocsRedis: scripting with Lua
• Docsredis4cats
• DocsCats Effect: getting started
• ArticleStripe: scaling your API with rate limiters