Studying CNTK (5): ConvNet_CIFAR10.cntk and ConvNet_CIFAR10_DataAug.cntk
In terms of tasks, object recognition is the hardest, because object size and appearance change flexibly. Traditional neural networks find this almost impossible, while convolutional technology overcomes this—though it still depends on massive compute.
The model in ConvNet is:
model = Sequential (
Normalize {featMean, featScale} :
ConvolutionalLayer {64, (3:3), pad = true} : ReLU :
ConvolutionalLayer {64, (3:3), pad = true} : ReLU :
MaxPoolingLayer {(3:3), stride = (2:2)} :
ConvolutionalLayer {64, (3:3), pad = true} : ReLU :
ConvolutionalLayer {64, (3:3), pad = true} : ReLU :
MaxPoolingLayer {(3:3), stride = (2:2)} :
DenseLayer {256} : Dropout : ReLU :
DenseLayer {128} : Dropout : ReLU :
LinearLayer {labelDim}
)
Because Sequential is used, layers can be connected directly with ":". The steps can be briefly described as: input normalization, two convolutions, pooling, two convolutions, pooling, fully connected 256, fully connected 128, output.
Not much different from before; running it on a machine takes about seven or eight minutes. Results after running:
Minibatch[1-100]: errs = 17.563% * 1600; top5Errs = 0.750% * 1600
Minibatch[101-200]: errs = 19.563% * 1600; top5Errs = 1.438% * 1600
Minibatch[201-300]: errs = 17.188% * 1600; top5Errs = 1.875% * 1600
Minibatch[301-400]: errs = 17.875% * 1600; top5Errs = 0.813% * 1600
Minibatch[401-500]: errs = 18.313% * 1600; top5Errs = 0.875% * 1600
Minibatch[501-600]: errs = 17.938% * 1600; top5Errs = 1.000% * 1600
Minibatch[601-625]: errs = 20.000% * 400; top5Errs = 1.250% * 400
Final Results: Minibatch[1-625]: errs = 18.150% * 10000; top5Errs = 1.130% * 10000
Error rate is about 18%, accuracy about 82%.
Using ConvNet_CIFAR10_DataAug.cntk for 80 iterations—though unexpectedly the disk filled up before it finished—accuracy only reached 85.156%. Runtime was much longer; the improvement can only be called fair.
Starting minibatch loop.
(GPU): creating curand object with seed 148
(GPU): creating curand object with seed 149
Epoch[75 of 80]-Minibatch[ 1- 100, 14.29%]: ce = 0.43693989 * 6400; errs = 14.578% * 6400; time = 2.2431s; samplesPerSecond = 2853.3
Epoch[75 of 80]-Minibatch[ 101- 200, 28.57%]: ce = 0.44393230 * 6400; errs = 14.703% * 6400; time = 2.1941s; samplesPerSecond = 2916.9
Epoch[75 of 80]-Minibatch[ 201- 300, 42.86%]: ce = 0.45336273 * 6400; errs = 14.797% * 6400; time = 2.2024s; samplesPerSecond = 2906.0
Epoch[75 of 80]-Minibatch[ 301- 400, 57.14%]: ce = 0.46737686 * 6400; errs = 15.250% * 6400; time = 2.1987s; samplesPerSecond = 2910.8
Epoch[75 of 80]-Minibatch[ 401- 500, 71.43%]: ce = 0.45469894 * 6400; errs = 15.000% * 6400; time = 2.1968s; samplesPerSecond = 2913.3
Epoch[75 of 80]-Minibatch[ 501- 600, 85.71%]: ce = 0.46115814 * 6400; errs = 15.000% * 6400; time = 2.1925s; samplesPerSecond = 2919.0
Epoch[75 of 80]-Minibatch[ 601- 700, 100.00%]: ce = 0.45480957 * 6400; errs = 14.734% * 6400; time = 2.1945s; samplesPerSecond = 2916.4
Finished Epoch[75 of 80]: [Training] ce = 0.45361355 * 50000; errs = 14.844% * 50000; totalSamplesSeen = 3750000; learningRatePerSample = 1.5625001e-005; epochTime=17.2243s
Worth noting: in ConvNet_CIFAR10_DataAug.cntk, ImageReader is used instead of converting all images into text data; text reading uses CNTKTextFormatReader.
reader = {
verbosity = 0 ; randomize = true
deserializers = ({
type = "ImageDeserializer" ; module = "ImageReader"
file = "$dataDir$/train_map.txt"
input = {
features = { transforms = (
{ type = "Crop" ; cropType = "random" ; cropRatio = 0.8 ; jitterType = "uniRatio" } :
{ type = "Scale" ; width = 32 ; height = 32 ; channels = 3 ; interpolations = "linear" } :
{ type = "Mean" ; meanFile = "$dataDir$/CIFAR-10_mean.xml" } :
{ type = "Transpose" }
)}
labels = { labelDim = 10 }
}
})
}