Image Classification API

Project roadmap

1. 01

   Set up the project and download the model

   ~1.5h

   Create a Python 3.11+ project with fastapi, uvicorn, onnxruntime, numpy, Pillow, pandas, python-multipart and pytest. Download a pretrained MobileNetV2 or ResNet-50 ONNX model and the ImageNet labels file from the ONNX Model Zoo into models/, and record their source and license in the README. Done when: a script loads the model with onnxruntime.InferenceSession and prints its input shape.

2. 02

   Preprocess images with Pillow

   ~2h

   Write preprocess(image_bytes) that opens the image with Pillow, applies the EXIF orientation (ImageOps.exif_transpose), converts it to RGB, resizes the shorter side to 256, center-crops to 224×224, and converts it to a float32 numpy array normalized with the ImageNet mean and standard deviation, in the channel order the model expects (NCHW). Done when: a test image produces an array of shape (1, 3, 224, 224) with values in the expected range.

3. 03

   Run the model and rank labels

   ~1.5h

   Write classify(image_bytes, top_k=5) that preprocesses the image, runs the session, applies softmax to the output scores, and returns the top k labels with probabilities, sorted from highest to lowest. Load the session once and reuse it. Done when: a photo of a dog returns a dog breed as the top label with a high probability.

4. 04

   Build the API

   ~2h

   Create a FastAPI app with POST /classify, accepting a file upload and an optional top_k (1 to 10), and GET /health. Reject files over 5 MB (413), files that aren't JPEG, PNG or WebP (415), and images Pillow can't open (400), with clear JSON errors. Return the labels, the probabilities and the processing time in ms. Done when: uploading an image with curl returns ranked labels, and invalid files get the right errors.

5. 05

   Limit concurrency and cache results

   ~2.5h

   Run inference in a thread pool (run_in_threadpool) so the server stays responsive, and cap concurrent inferences with an asyncio.Semaphore (for example 4). When the limit is reached, requests wait up to 2 seconds, then get 503 with a Retry-After header. Cache results by the SHA-256 of the uploaded bytes in an LRU cache of 500 entries. Done when: 20 simultaneous requests never run more than 4 inferences at once, and a repeated image is served from the cache.

6. 06

   Add metrics and logging

   ~1.5h

   Add GET /metrics returning counts of requests, errors by type and cache hits, and the average and 95th percentile inference time. Log one structured JSON line per request with the image size, status, time and top label, never the image itself. Done when: after sending a few requests, /metrics reports correct numbers.

7. 07

   Evaluate and test

   ~2.5h

   Write evaluate.py that runs a folder of labeled images (for example a small subset of Imagenette) through classify, collects results in a pandas DataFrame, and prints top-1 and top-5 accuracy and latency percentiles. Write pytest tests with FastAPI's TestClient for a valid image, an oversized file, an unsupported type, a corrupt image and the cache. Done when: evaluate.py prints the accuracy table and pytest passes.

Resources

• DocsONNX Runtime Python API
• DocsONNX Model Zoo
• DocsPillow documentation
• DocsFastAPI: request files
• DocsImagenette dataset