# Python Lasagne tutorial: validation error lower than training error

In the Lasagne tutorial (here and source code here) a simple multilayer perceptron is trained over the MNIST dataset. The data is split in a training set and a validation set, and the training calculates the validation error on each epoch expressed as the average cross-entropy error per batch.

However, the validation error is always lower than the training error. Why does that happen? Shouldn't the training error be lower since it the data that the network is trained on? Could this be a result of the dropout layers (enabled during training, but disabled during validation error calculation)?

Output of the first few epochs:

Epoch 1 of 500 took 1.858s
training loss:                1.233348
validation loss:              0.405868
validation accuracy:          88.78 %
Epoch 2 of 500 took 1.845s
training loss:                0.571644
validation loss:              0.310221
validation accuracy:          91.24 %
Epoch 3 of 500 took 1.845s
training loss:                0.471582
validation loss:              0.265931
validation accuracy:          92.35 %
Epoch 4 of 500 took 1.847s
training loss:                0.412204
validation loss:              0.238558
validation accuracy:          93.05 %

• You are using an improper accuracy scoring rule which brings a lot of randomness to the evaluation. This is also a symptom of having training and test sets that are too small. How many independent observations are in each? – Frank Harrell Oct 23 '15 at 15:18
• What would be a better scoring rule? The training set has a size of 50000 observations and the validation 10000. – Cantfindname Oct 23 '15 at 15:29
• The validation sample is a bit on the low side. You might considering combining all the data and using resampling (e.g. bootstrap). Check out the Brier score as a good example of a proper scoring rule. – Frank Harrell Oct 23 '15 at 19:25