Validation Error less than training error? I found two questions here and here about this issue but there is no obvious answer or explanation yet.I enforce the same problem where the validation error is less than training error in my Convolution Neural Network. What does that mean?
 A: I don't have enough points to comment on @D-K's answer, but this is now answered as a FAQ on Keras' documentation:

Why is my training loss much higher than my testing loss?
A Keras model has two modes: training and testing. Regularization mechanisms, such as Dropout and L1/L2 weight regularization, are turned off at testing time. They are reflected in the training time loss but not in the test time loss.
Besides, the training loss that Keras displays is the average of the losses for each batch of training data, over the current epoch. Because your model is changing over time, the loss over the first batches of an epoch is generally higher than over the last batches. This can bring the epoch-wise average down. On the other hand, the testing loss for an epoch is computed using the model as it is at the end of the epoch, resulting in a lower loss.

A: I got similar results (test loss was significantly lower than training loss).  Once I removed the dropout regularization, both the loss became almost equal.
A: One possibility: If you are using dropout regularization layer in your network, it is reasonable that the validation error is smaller than training error. Because usually dropout is activated when training but deactivated when evaluating on the validation set. You get a more smooth (usually means better) function in the latter case.
A: my 2 cents: 
I also had the same problem even without having dropout layers.
In my case - batch-norm layers were the culprits.
When I deleted them - training loss became similar to validation loss.
Probably, that happened because during training batch-norm
uses mean and variance of the given input batch, which might be different
from batch to batch. But during evaluation batch-norm uses running mean and variance, both of which reflect properties of the whole training set much better than mean and variance of a single batch during training.
At least, that is how batch-norm is implemented in pytorch
A: Another possibility that combines both the answer of @cdeterman and @D-K in some way is if you're using some data augmentation mechanism. Infact data augmentation is usually done only on training set and not on validation set (as for the dropout regularization), and this may lead to a validation set containing "easier" cases to predict than those in the training set.
A: It is difficult to be certain without knowing your actual methodology (e.g. cross-validation method, performance metric, data splitting method, etc.).
Generally speaking though, training error will almost always underestimate your validation error.  However it is possible for the validation error to be less than the training.  You can think of it two ways:


*

*Your training set had many 'hard' cases to learn

*Your validation set had mostly 'easy' cases to predict


That is why it is important that you really evaluate your model training methodology.  If you don't split your data for training properly your results will lead to confusing, if not simply incorrect, conclusions.
I think of model evaluation in four different categories:


*

*Underfitting – Validation and training error high

*Overfitting – Validation error is high, training error low

*Good fit – Validation error low, slightly higher than the training error

*Unknown fit - Validation error low, training error 'high'
I say 'unknown' fit because the result is counter intuitive to how machine learning works.  The essence of ML is to predict the unknown.  If you are better at predicting the unknown than what you have 'learned', AFAIK the data between training and validation must be different in some way.  This could mean you either need to reevaluate your data splitting method, adding more data, or possibly changing your performance metric (are you actually measuring the performance you want?).
EDIT
To address the OP's reference to a previous python lasagne question.
This suggests that you have sufficient data to not require cross-validation and simply have your training, validation, and testing data subsets.  Now, if you look at the lasagne tutorial you can see that the same behavior is seen at the top of the page.  I would find it hard to believe the authors would post such results if it was strange but instead of just assuming they are correct let's look further.  The section of most interest to us here is in the training loop section, just above the bottom you will see how the loss parameters are calculated.
The training loss is calculated over the entire training dataset.  Likewise, the validation loss is calculated over the entire validation dataset.  The training set is typically at least 4 times as large as the validation (80-20).  Given that the error is calculated over all samples, you could expect up to approximately 4X the loss measure of the validation set.  You will notice, however, that the training loss and validation loss are approaching one another as training continues.  This is intentional as if your training error begins to get lower than your validation error you would be beginning to overfit your model!!!
I hope this clarifies these errors.  
A: @cdeterman and @D-K have good explanation. I would like to one more reason - data leakage. Some part of your train-data are "closely related" with the test-data. 
Potential example: imagine you have 1000 dogs and 1000 cats with 500 similar pictures per pet (some owners love to take pictures of their pets in very similar positions), say on the background. So if you do random 70/30 split, you'll get data leakage of train data into the test data.
A: A lower validation than training error can be caused by fluctuations associated with dropout or else, but if it persists in the long run this may indicate that the training and validation datasets were not actually drawn from the same statistical ensembles. This could happen if your examples come from a series and if you did not properly randomize the training and validation datasets.
A: Simply put, if training loss and validation loss are computed correctly, it is impossible for training loss to be higher than validation loss. This is because back-propagation DIRECTLY reduces error computed on the training set and only INDIRECTLY (not even guaranteed!) reduces error computed on the validation set. 
There must be some additional factors that are different while training and while validating. Dropout is a good one, but there can be others. Make sure to check the documentation of whatever library that you are using. Models and layers can usually have default settings that we don't commonly pay attention to.
