What could be the reasons that making validation loss jumping up and down?

I am building some image classification model with reasonable size data (~3K) images in both training and validation set. However, I noticed the performance on validation set is not stable.

For example, here are outputs over 10 epochs (acc means accuracy binary classification on balanced data)

epoch [1]: training loss: 2.27 - acc: 0.50 - val_loss: 3.29 - val_acc: 0.49
epoch [2]: training loss: 1.55 - acc: 0.50 - val_loss: 0.92 - val_acc: 0.50
epoch [3]: training loss: 1.07 - acc: 0.51 - val_loss: 1.43 - val_acc: 0.53
epoch [4]: training loss: 0.87 - acc: 0.58 - val_loss: 1.85 - val_acc: 0.61
epoch [5]: training loss: 0.59 - acc: 0.72 - val_loss: 0.58 - val_acc: 0.61
epoch [6]: training loss: 0.52 - acc: 0.79 - val_loss: 2.30 - val_acc: 0.50
epoch [7]: training loss: 0.38 - acc: 0.85 - val_loss: 0.17 - val_acc: 0.86
epoch [8]: training loss: 0.32 - acc: 0.88 - val_loss: 1.52 - val_acc: 0.60
epoch [9]: training loss: 0.21 - acc: 0.91 - val_loss: 0.14 - val_acc: 0.88
epoch [10]: training loss: 0.34 - acc: 0.88 - val_loss: 2.81 - val_acc: 0.49


We can see that in training, it seems fine, but for epoch 6 and 8 validation loss was very high, and the final epoch 10, the validation loss got so high that the model become useless.

What could be the reason causing this? If it is overfitting on training data, why we are not seeing steady increase on validation loss?

• Please add a description of how the 10 epochs differ. – user332577 Jul 20 at 12:29

Tracking the training loss within epochs could confirm or disconfirm this hypothesis. (As an aside, measuring training statistics every mini-batch could consume too much memory if you have a large dataset and/or small mini-batch size. So instead, every $$k > 1$$ mini-batches, record two pieces of data:
2. the mean of the most recent $$k$$ mini-batches. Choose the smallest $$k$$ that doesn't consume too much memory. )