In the Lasagne tutorial (here and source code here) a simple multilayer perceptron is trained over the MNIST dataset. The data is split in a training set and a validation set, and the training calculates the validation error on each epoch expressed as the average cross-entropy error per batch.
However, the validation error is always lower than the training error. Why does that happen? Shouldn't the training error be lower since it the data that the network is trained on? Could this be a result of the dropout layers (enabled during training, but disabled during validation error calculation)?
Output of the first few epochs:
Epoch 1 of 500 took 1.858s
training loss: 1.233348
validation loss: 0.405868
validation accuracy: 88.78 %
Epoch 2 of 500 took 1.845s
training loss: 0.571644
validation loss: 0.310221
validation accuracy: 91.24 %
Epoch 3 of 500 took 1.845s
training loss: 0.471582
validation loss: 0.265931
validation accuracy: 92.35 %
Epoch 4 of 500 took 1.847s
training loss: 0.412204
validation loss: 0.238558
validation accuracy: 93.05 %