In the Lasagne tutorial (here and source code here) a simple multilayer perceptron is trained over the MNIST dataset. The data is split in a training set and a validation set, and the training calculates the validation error on each epoch expressed as the average cross-entropy error per batch.
However, the validation error is always lower than the training error. Why does that happen? Shouldn't the training error be lower since it the data that the network is trained on? Could this be a result of the dropout layers (enabled during training, but disabled during validation error calculation)?
Output of the first few epochs:
Epoch 1 of 500 took 1.858s training loss: 1.233348 validation loss: 0.405868 validation accuracy: 88.78 % Epoch 2 of 500 took 1.845s training loss: 0.571644 validation loss: 0.310221 validation accuracy: 91.24 % Epoch 3 of 500 took 1.845s training loss: 0.471582 validation loss: 0.265931 validation accuracy: 92.35 % Epoch 4 of 500 took 1.847s training loss: 0.412204 validation loss: 0.238558 validation accuracy: 93.05 %