Webhook Fan-Out Service
Build a webhook fan-out service in Go: other systems send it an event once, and it delivers that event to every subscriber interested in it (such as partner systems or internal services), each through its own HTTP call. Every delivery has its own state and retry schedule, so one slow or broken subscriber never delays the others; payloads are signed so subscribers can verify they came from you, and failures are retried with increasing waits until they succeed or give up. Events are accepted over HTTP, and subscriptions are managed through a small gRPC admin API. Delivery state is stored so nothing is lost on restart. You'll learn fan-out and per-target delivery, retries and backoff, HMAC signatures, gRPC, and testing with fake subscribers.
- Estimate
- ~11.5h
- Steps
- 6
- Completed by
- 0
- Proposed by
- codeseed.app
net/http · gRPC
Project roadmap
- 01
Define the system
~1.5hWrite DESIGN.md: subscriptions (id, target URL, event types, secret and active flag), events (id chosen by the sender for idempotency, type, payload and received time), deliveries (event, subscription, status pending, delivered or failed, attempts, next attempt time and last error), the signature header (HMAC-SHA256 of timestamp plus body), the retry schedule, and what happens on restart. Done when: DESIGN.md covers every entity and rule.
- 02
Accept events and create deliveries
~2hBuild POST /events with net/http, authenticated with an API key, validating the JSON and its size. In one transaction (using SQLite through database/sql, or PostgreSQL), store the event (ignoring a repeated event id) and create a delivery for each active subscription matching the event type. Return 202 with the number of deliveries. Done when: an event creates one delivery per matching subscriber, and resending it creates nothing new.
- 03
Deliver with per-target retries
~2.5hRun a pool of workers that claim due deliveries, POST the payload to the subscriber with a timeout, the signature and a delivery id header, and mark success on any 2xx. On failure, schedule the next attempt with exponential backoff and jitter, up to a maximum, then mark it failed. Limit concurrent deliveries per subscriber, so a slow one can't use every worker. Done when: a failing subscriber is retried on schedule while others receive events immediately.
- 04
Manage subscriptions with gRPC
~2hDefine an admin service in a .proto file (create, update, pause, delete and list subscriptions; list deliveries for an event; and redeliver a failed delivery), generate the Go code, and implement it with validation of URLs and event types. Generate secrets on creation and show them once. Done when: subscriptions can be managed with grpcurl or a small client.
- 05
Survive restarts and handle edge cases
~1.5hBecause deliveries are stored, a restart simply resumes pending ones; make sure deliveries claimed by a crashed worker are released after a lease timeout. Handle subscribers that redirect (don't follow redirects to other hosts), that return huge responses (read only a little), and that are paused (keep their deliveries pending). Done when: killing the service mid-delivery and restarting it delivers everything.
- 06
Test it
~2hWrite tests with httptest.Server subscribers that succeed, fail a few times, time out or always fail: fan-out to the right subscribers, idempotent events, retries and backoff (with a controllable clock), per-subscriber isolation, signature verification from the subscriber's side, and recovery after a restart. Run with -race. Done when: go test -race ./... passes.
Resources
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack