Studying CNTK (Part IV): ConvNet_MNIST.cntk

The results show:

Minibatch[1-10]: errs = 0.470% * 10000; ce = 0.01797164 * 10000
Final Results: Minibatch[1-10]: errs = 0.470% * 10000; ce = 0.01797164 * 10000; perplexity = 1.01813410

The error rate is only 0.47%, which corresponds to an accuracy of 99.53% — that is very high indeed.

This is the result of using three convolutional layers, with 40 iterations; it took quite a long time to run. According to the MNIST website, the best published result is 0.23, using 35 convolutional layers in total — it is hard to imagine how long that would take to compute.

The code is almost the same as before; the main difference is in the model, which looks like this:

 model = Sequential (
 Scale {featScale} :
 ConvolutionalLayer {32, (5:5), pad = true} : ReLU :
 MaxPoolingLayer {(3:3), stride=(2:2)} :
 ConvolutionalLayer {48, (3:3), pad = false} : ReLU :
 MaxPoolingLayer {(3:3), stride=(2:2)} :
 ConvolutionalLayer {64, (3:3), pad = false} : ReLU :
 DenseLayer {96} : Dropout : ReLU :  
 LinearLayer {labelDim}
 )

This is the classic architecture: input, normalization, convolution, pooling, convolution, pooling, convolution, pooling, fully connected, output.

Note also that 32 filters are used during convolution, which further strengthens the ability to extract features, and that pooling uses (3:3) with stride=(2:2).