Distributed Rate Limiter

Project roadmap

1. 01

   Design the limiter

   ~2h

   Write DESIGN.md: the buckets table (key, tokens as a decimal, last refill time and the policy), how refilling works (tokens + elapsed seconds × rate, capped at capacity), the single UPDATE ... RETURNING statement that refills and takes tokens atomically, how time is taken from the database (now()) so server clocks don't matter, the policy file in JSON, and what happens when the database is slow or down. Done when: DESIGN.md explains the algorithm, the SQL and failure behavior.

2. 02

   Implement the token bucket

   ~2h

   Implement the token bucket as a pure Java class first (state, now and cost to decision and new state) and test it: refilling, bursts up to capacity, never exceeding capacity and fractional tokens. Load policies (per plan: capacity and refill per second) from a JSON file with Jackson, validating them. Done when: unit tests of the pure bucket and policy loading pass.

3. 03

   Store buckets in PostgreSQL

   ~3h

   Create the table with a migration, and write the atomic take operation: an INSERT ... ON CONFLICT DO UPDATE (or UPDATE with a CTE) that refills based on now(), takes the tokens only if enough are available, and returns whether it was allowed and the remaining tokens, all in one statement using JDBC with a connection pool (HikariCP). Done when: the database version gives the same decisions as the pure bucket.

4. 04

   Expose the limiter

   ~1.5h

   Expose a RateLimiter interface (tryAcquire(key, cost) returning a decision with remaining tokens and retry-after), and a servlet filter or HTTP endpoint that applies it and returns 429 with Retry-After and rate limit headers. Done when: a small demo service rejects requests over the limit with correct headers.

5. 05

   Reduce database load

   ~2h

   Add an optional local mode where each instance reserves a small batch of tokens from the database at once and hands them out from memory, returning unused ones or letting them expire, and measure how much this reduces database calls and how much it can overshoot the limit. Add a timeout on database calls and a configurable fallback (allow, deny, or a local limit). Done when: README.md shows database calls and accuracy with and without batching.

6. 06

   Test with many instances

   ~2.5h

   Write tests with Testcontainers PostgreSQL: many threads across two limiter instances calling tryAcquire for the same key at once never get more tokens than the bucket allows, refilling over time, the fallback when the database is stopped, and the batching mode's worst-case overshoot. Run a small load test and record throughput and latency. Done when: the tests pass, and README.md shows the load test results.

Resources

• ArticleToken bucket (Wikipedia)
• DocsPostgreSQL: INSERT ... ON CONFLICT
• DocsJackson databind
• DocsHikariCP
• DocsTestcontainers for Java