Is it a good idea to continue training a model after the train/validation accuracy has stopped improving?

Question

The following animated diagram shows the training statistics of a Deep Neural Network classifier at the end of each epoch:

The diagrams on the left show the accuracy (upper) and loss (lower) values on training and validation data per epoch. The diagrams on the right show the distribution of confidence values (i.e. maximum of softmax scores) in training (upper) and validation (lower) data.

As you can see, both of training and validation accuracy values stop improving after a certain epoch (i.e. epoch #25) and reach a plateau. However, the confidence of (correct) predictions keeps increasing as we continue the training, while the training and validation loss value still has a decreasing trend (which is also consistent with that). Now:

• Is it safe to claim that the model's predictions are much more confident in, say, epoch #50 compared to those of epoch #25 and therefore it's a better model to use? (My own answer to that is yes, because that effect is also happening with the held-out validation set.)
• Is this a good approach - i.e. continuing training after reaching the accuracy plateau, while keeping an eye on loss value - to get a model with much more confident predictions, especially in applications where not only the correctness of prediction is important but also the confidence of the predictions is of great importance? Or is there better alternatives to this? (For example, as one point, I can see a trade-off of training time/computational resources vs. higher confidence.)

Answer 1

In some settings, it would be fine to keep training the network, and the validation loss will continue decreasing towards an asymptote. However, boosting the model's confidence is a poor motivation for continuing the training.

First, it is unclear whether your model becomes more calibrated as you keep training. Typically, these deep neural networks just become over-confident without correctly capturing their probability of correct classification. To check this, you have to use a calibration plot or measures such as Expected Calibration Error (ECE).

Secondly, there are ways of adjusting your model's confidence that are both computationally cheaper and more principled. In general, these calibration methods operate by scaling the confidence via the adjustment of few parameters (or even a single parameter) in the readout layer such that some cost function defined on held-out data is minimized. This function can be cross-entropy if your aim is well-calibrated models.

Reference:

Guo, Chuan, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. "On calibration of modern neural networks." arXiv preprint arXiv:1706.04599 (2017).