What this is

This is a neural network classifier for handwritten digits (trained with the MNIST dataset[0]). It's implemented as a feedforward neural network and aims to be a lean-and-mean demo of the concepts, hopefully hackable and easy to play with. Provides ~98% accuracy in about 18 seconds on an M1 Ultra.

The meat is in model.rs.

What this does

First, a neural network is trained, then it is given handwritten digits from another dataset to recognize. Finally, the digits are printed out, along with some numbers to provide insight into how the training went.

How to run this stuff

cargo run --release is how to run it so no BLAS, GPU, or C libraries

git clone https://github.com/kromych/digit-recognizer
cd digit-recognizer
cargo run --release

The data/*.gz files are read as they are, no unpacking step and nothing for Windows users to figure out. Plain uncompressed IDX files will work, too, if you happen to have them lying around.

2026-07-26T17:12:43.861547Z  INFO digit_recognizer: Training on 55000 samples, validating on 5000, testing on 10000
2026-07-26T17:12:43.862446Z  INFO digit_recognizer: Training 30 epochs, batch 32, learning rate 0.1, Relu hidden layer of 128
2026-07-26T17:12:44.439109Z  INFO digit_recognizer::model: epoch 1/30: loss 0.3282, validation accuracy 93.64%
2026-07-26T17:12:45.022432Z  INFO digit_recognizer::model: epoch 2/30: loss 0.1565, validation accuracy 95.88%
....
2026-07-26T17:13:00.836443Z  INFO digit_recognizer::model: epoch 29/30: loss 0.0038, validation accuracy 97.90%
2026-07-26T17:13:01.422276Z  INFO digit_recognizer::model: epoch 30/30: loss 0.0036, validation accuracy 97.96%
2026-07-26T17:13:01.422286Z  WARN digit_recognizer: Validation accuracy peaked at 98.06% on epoch 27 and finished at 97.96%; try fewer epochs, or --load a model saved earlier.
2026-07-26T17:13:01.481605Z  INFO digit_recognizer: Accuracy: 98.02%

and then observe something like and like that:

╭───────────────────────────────╮
│             info              │
├───────────────────────────────┤
│ Sample 0: true 7, predicted 7 │
│ ✓ Correct prediction!        │
╰───────────────────────────────╯

    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @@@@@@*:-=#%@@@@@@@@@@@@@@@@
    @@@@@@.     ::::::::-#@@@@@@
    @@@@@@#+*+-. .    .  =@@@@@@
    @@@@@@@@@@@%#@####%. +@@@@@@
    @@@@@@@@@@@@@@@@@@* :%@@@@@@
    @@@@@@@@@@@@@@@@@%. *@@@@@@@
    @@@@@@@@@@@@@@@@@= .#@@@@@@@
    @@@@@@@@@@@@@@@@#  #@@@@@@@@
    @@@@@@@@@@@@@@@@= :@@@@@@@@@
    @@@@@@@@@@@@@@@@: #@@@@@@@@@
    @@@@@@@@@@@@@@@+ -@@@@@@@@@@
    @@@@@@@@@@@@@@* .#@@@@@@@@@@
    @@@@@@@@@@@@@%. -@@@@@@@@@@@
    @@@@@@@@@@@@@: .%@@@@@@@@@@@
    @@@@@@@@@@@@%  *@@@@@@@@@@@@
    @@@@@@@@@@@%. +@@@@@@@@@@@@@
    @@@@@@@@@@@=  #@@@@@@@@@@@@@
    @@@@@@@@@@#   #@@@@@@@@@@@@@
    @@@@@@@@@@+  .%@@@@@@@@@@@@@
    @@@@@@@@@@+ :%@@@@@@@@@@@@@@
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@

..................... more stuff .................................................

Per-class performance:

┌───────────┬───────────┬────────┬──────────┬─────────┬─────────┐
│   Class   │ Precision │ Recall │ F1-Score │ Correct │ Support │
├───────────┼───────────┼────────┼──────────┼─────────┼─────────┤
│ 0         │ 0.981     │ 0.989  │ 0.985    │ 969     │ 980     │
├───────────┼───────────┼────────┼──────────┼─────────┼─────────┤
....
├───────────┼───────────┼────────┼──────────┼─────────┼─────────┤
│ Macro avg │ 0.980     │ 0.980  │ 0.980    │ 9802    │ 10000   │
└───────────┴───────────┴────────┴──────────┴─────────┴─────────┘

Overall accuracy: 98.02%

If your terminal speaks Sixel graphics, add -g for more fun! The program asks the terminal directly rather than guessing from $TERM, so -g on a terminal that cannot do it falls back to ASCII instead of spraying escape codes across your screen.

Training takes long enough to be annoying if you are only fiddling with the display, so the weights can be kept:

cargo run --release -- --save mnist.model    # train once
cargo run --release -- --load mnist.model    # ...reuse forever

There are more arguments to play with:

cargo run --release -- --help

The interesting ones are --hidden-size, --batch-size, --learning-rate, --epochs and --activation relu|sigmoid. A bigger hidden layer buys a little: -H 256 gets to 98.15% and takes twice as long.

Neural network

Input Layer (784) -> Hidden Layer (ReLU)   -> Output Layer (Softmax)
28 x 28 pixels       128 neurons (default)    10 neurons (digits 0-9)

Mathematics behind the scenes

A batch of $B$ samples is a matrix $X$ with one sample per column, which turns every step below into a matrix product.

Forward propagation

• Input: 784 rows of pixels normalized to $[0, 1]$
• Hidden layer, with the bias broadcast across the batch:

$$Z_1 = W_1 X + b_1, \qquad A_1 = \mathrm{ReLU}(Z_1)$$

• Output layer, softmax taken independently per column:

$$Z_2 = W_2 A_1 + b_2, \qquad A_2 = \mathrm{softmax}(Z_2)$$

• Weight initialization: He[1] for ReLU, Xavier[4] for sigmoid. These are not interchangeable. He assumes the non-linearity throws away half the variance, which sigmoid does not do.

Loss function

Categorical cross-entropy, averaged over the samples in the batch:

$$L = -\frac{1}{B} \sum_{b} \sum_{i} Y_{ib} \log (A_2)_{ib}$$

Backpropagation [2]

The softmax and cross-entropy Jacobians cancel, which is the reason those two are paired:

$$dZ_2 = \frac{A_2 - Y}{B}, \qquad dW_2 = dZ_2 A_1^{\top}, \qquad db_2 = dZ_2 \mathbf{1}$$ $$dZ_1 = (W_2^{\top} dZ_2) \odot \mathrm{ReLU}'(Z_1), \qquad dW_1 = dZ_1 X^{\top}, \qquad db_1 = dZ_1 \mathbf{1}$$

Weights are updated by mini-batch stochastic gradient descent[3], with the training set reshuffled between epochs.

If you change any of this, cargo test checks the analytic gradients above against central differences of the loss, and for both activations.

Some limitations

• One hidden layer. Its size is the only architectural knob
• Plain SGD so no momentum, no adaptive learning rate, no weight decay. Hence, the result is somewhat sensitive to --learning-rate
• No convolutions, which is what stands between this and 99%+!
• No GPU acceleration!
• Held-out validation is reported but not acted on. You just get a warning when accuracy peaked early.

Wouldn't be possible without

0. MNIST dataset of 60,000 training images (28 x 28 pixels, grayscale) and 10,000 test images: LeCun, Y., Cortes, C., & Burges, C. J. C. The MNIST Database of Handwritten Digits, 1998
1. Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
2. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. Learning representations by back-propagating errors, Nature, 1986
3. Stochastic gradient descent
4. Glorot, X., & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks, AISTATS 2010
5. Cross-entropy
6. VT330/VT340 Programmer Reference Manual, chapter 14, "Sixel Graphics", for sixel.rs

THANK YOU!!!