Feedforward Neural Network from Scratch
Build a small neural network in plain C, with no libraries except math.h, and train it to recognize patterns, first on a tiny problem (XOR), then on a real dataset such as handwritten digits or the Iris flowers. You'll write everything yourself: matrix operations, the forward pass that turns inputs into predictions, backpropagation, which calculates how each weight should change to reduce the error, and gradient descent, which applies those changes. You'll check your gradients numerically so you know the math is right, then watch the loss go down and the accuracy go up. You'll learn how neural networks really learn, numerical programming in C, memory management, and debugging math-heavy code.
- Estimate
- ~11.5h
- Steps
- 6
- Completed by
- 0
- Proposed by
- codeseed.app
math.h · manual matrix operations
Project roadmap
- 01
Write matrix operations
~2hCreate a Matrix struct (rows, columns and a flat array of doubles) with functions to create and free matrices, multiply two matrices, add a bias to each row, apply a function to every element, and transpose. Check dimensions in every function and fail with a clear message on a mismatch. Done when: unit tests (with assert) confirm each operation on small hand-computed examples, and valgrind reports no leaks.
- 02
Build the network structure
~1.5hDefine a Layer struct (weights, biases, and space for activations and gradients) and a Network made of layers with sizes given at creation (for example 2, 4 and 1 neurons). Initialize weights with small random values scaled by the layer size (Xavier or He initialization), with a fixed seed so runs are repeatable. Done when: a network is created and freed cleanly with the chosen layer sizes.
- 03
Implement the forward pass
~1.5hCompute each layer's output as the input times the weights plus the bias, followed by an activation: sigmoid or ReLU for hidden layers, and sigmoid (for yes or no outputs) or softmax (for several classes) at the end. Write the loss functions: mean squared error and cross-entropy. Done when: the forward pass gives outputs of the right size, and the loss is computed.
- 04
Implement backpropagation
~3hCompute the error at the output, then move backwards through the layers, applying the chain rule to get each weight's and bias's gradient, using the derivative of each activation. Update the weights with gradient descent using a learning rate. Verify your gradients with a numerical check: change one weight slightly, measure the change in loss, and compare with your computed gradient. Done when: the numerical and computed gradients agree closely, and the network learns XOR.
- 05
Load a real dataset
~1.5hWrite a CSV loader for a small real dataset (for example the Iris dataset, or MNIST digits from their CSV version), normalizing the input values and one-hot encoding the labels. Shuffle the data with a seeded random generator and split it into training and test sets. Handle bad lines with clear errors. Done when: the dataset loads into input and label matrices.
- 06
Train and evaluate
~2hTrain in mini-batches for several epochs, printing the training loss and test accuracy after each epoch. Try different learning rates and hidden layer sizes, and record the results in README.md. Save the trained weights to a file and load them again to classify new inputs. Done when: the network reaches a good accuracy on the test set (for example over 90% on Iris), and README.md shows the experiments.
Ready to build this?
Get a GitHub repo and start building. Your AI reviewer checks each step as you go.
Tech stack