Distributed Task Queue with Celery and Redis

Project roadmap

1. 01

   Set up Celery with a Redis broker

   ~1.5h

   Configure a Celery app connected to a local Redis instance and define a simple test task.

2. 02

   Build real background tasks

   ~2h

   Implement tasks such as image resizing or report generation that take meaningful time to run.

3. 03

   Add retries and error handling

   ~1.5h

   Configure automatic retries with backoff for transient failures and dead-letter handling for permanent ones.

4. 04

   Add scheduled periodic tasks

   ~1.5h

   Use Celery beat to run a recurring cleanup or report-generation task on a schedule.

5. 05

   Expose an API to enqueue and check jobs

   ~2h

   Build a small Flask endpoint that enqueues tasks and another that reports job status by ID.

Resources

• DocsPython Documentation