A common technique to do early stopping is to split the data to 3 parts: train, validation and test, and train on train set. After each epoch (or every K epochs) of training we check the loss of the model on the validation set, and if this loss doesn't improve for a prespecified number epochs (the exact number is the patience hyperparameter), we stop and choose the model with the best validation loss so far. This algorithm is described in section 7.8 of the deep learning book.
Next, the book discusses two approaches to reuse the validation set to further improve the model. The first (7.2) is to discard the trained parameters and start training from scratch, this time on both train and validation, using the ideal number of epochs, i*, from the early stopping stage.
The second (7.3) is to keep the parameters and keep training
on both train and validation, till the loss on the validation set is less than the loss on train from the previous step.
Frankly, both approaches leave much to be desired for. 7.2 is wasteful by discarding the learned parameters on train. Besides there's no guarantee that the number of epochs, i*, is also optimal for both train and validation. As for 7.3, first of all, as the book says there's no guarantee for termination as the loss won't necessarily improve. Second, it seems like "self-cheating" if we evaluate on a subset of the training set (recall that we train on train and validation, and evaluate on validation).
Thus, I've thought of a third option: Take the best parameters from the early stopping stage (i.e. 7.1) and continue to train on train and validation. Stopping when the training loss doesn't improve for several epochs. Admittedly, we're interested in the validation loss (i.e. the generalization error) to prevent overfitting, but assuming that validation is much smaller than train, there will be less updates and thus the model will still be able to generalize. Does this approach make sense? Are there better alternatives?
Update: Yoshua Bengio, the author of the book and one of the founding fathers of modern deep learning, has replied regarding my concerns of "cheating" in 7.3:
You use valid for determining the stopping point while training on subtrain (which excludes valid). Once you have found a stopping point, though, there is nothing wrong in training (up to the stopping point) using the whole of train (= subtrain + valid).
