Validation loss decreasing faster than training loss I have two different scenarios that I ran across and I can't seem to wrap my head around what caused them.
In this scenario, my validation loss, in orange, initially fell faster than my training loss, in blue.

In this scenario, my training loss has all but converged but the validation loss continues to make big improvements.

The end results look look as expected and the final validation loss is still higher than the training loss. Does anyone have any insight into what might have caused this to happen? 
 A: Because validation data set is different from the training data set, so, ideally anything can happen in validation error over time.
For your first plot, we can see that training loss line is much smoother than validation line, this indicates you have a lot of training samples but relatively small validation sample (which is very normal in most cases). On the other hand, if number of validation samples is small, the validation loss is "less stable" and will have "large variance". This is why you may see sudden drop or increase over time. Let us use an extreme example to derive intuition: suppose there are only two instances in validation set, if the model can make one good classification, then you may see a sudden drop when it happens (when the training algorithm gets the parameters right for that instance).
For your second plot, we can see we got a good performance at training set very fast, but it performs not as good as training on validation set. This is some overfitting on the training set. Adding regularization will help to make the two lines closer but at the cost of both lines will have higher errors.
To conclude, this is may be because the validation set is small and not coming from the "same distribution" of the training set.
