CNN Inference Engine (Pure C++)

Project roadmap

1. 01

   Train and export a model

   ~2h

   In Python with PyTorch, train a small CNN on MNIST (for example two convolution layers with ReLU and max pooling, then two fully connected layers) to about 98% test accuracy. Export every layer's weights and biases to a simple binary file you define (a header with layer shapes, then float32 values in a documented order), and save 100 test images with PyTorch's outputs for later comparison. Done when: the weights file and reference outputs are saved.

2. 02

   Write the Tensor class

   ~2h

   Write a Tensor class holding a shape (for example channels, height, width) and the values in one contiguous std::vector<float>, with an index function that computes the position in memory, and bounds checks in debug builds. Add helpers to create, fill and compare tensors (with a tolerance). Done when: unit tests show indexing and comparison work.

3. 03

   Implement convolution

   ~2.5h

   Implement a Conv2D layer's forward pass with stride and padding: for each output channel and position, sum the input window multiplied by the kernel across all input channels, plus the bias. Make the weight layout match PyTorch's (out channels, in channels, kernel height, kernel width). Test it against small examples computed by hand and by PyTorch. Done when: your convolution matches PyTorch's output for the same input within 1e-5.

4. 04

   Implement the other layers

   ~2h

   Implement ReLU, MaxPool2D, Flatten (matching PyTorch's order), a Linear layer (a matrix times a vector plus bias) and softmax (subtracting the maximum first for numerical stability). Put the layers behind a common interface and chain them into a Model. Done when: each layer matches PyTorch on test inputs.

5. 05

   Load weights and run the network

   ~2h

   Write a loader that reads your weights file, checks the header and shapes against the model's layers (with clear errors on mismatch), and fills the layers. Load an MNIST image (from the saved test images or the original file format), run the full network, and print the predicted digit and probabilities. Done when: the predictions for all 100 saved images match PyTorch's outputs within a small tolerance.

6. 06

   Evaluate accuracy

   ~1.5h

   Run the engine on the full MNIST test set (10,000 images, read from the original IDX files) and report the accuracy, a confusion matrix, and a few misclassified examples. The accuracy must match PyTorch's. Done when: the accuracy matches the Python model's.

7. 07

   Make it faster

   ~2.5h

   Measure images per second with std::chrono, then optimize: reorder loops for better memory access, avoid allocating inside loops, build with -O3 -march=native, and try the im2col approach (turning convolution into matrix multiplication), or run images in parallel with threads. Record the speed after each change in README.md, and confirm the results never change. Done when: README.md shows the speedup for each optimization, with accuracy unchanged.

Resources

• DocsCS231n: convolutional neural networks
• DocsPyTorch: Conv2d
• DocsPyTorch: training a classifier tutorial
• ArticleMNIST database (Wikipedia)
• ArticleWhy GEMM is at the heart of deep learning (im2col)