I've read posts mentioning that after selecting your model with the validation set you train with the full set (training + validation set) and test the model on the test set.
However, what if the validation set was used for early stopping during training?
The training part might early stop at 12 epochs but that doesn't mean that it is the optimal amount of epochs for any training set in order to achieve generalization in unseen data.
Even worse, in my case, I am training a neural network where the amount the learning rate decreases in each iteration depends on the maximum number of epochs.
So even if I find that the optimal amount of epochs during training was 12, setting the maximum amount of epochs at 12 in order to retrain will yield a different result than early stopping at 12 when the maximum amount of epochs is 30.
So my question is:
Do I have to somehow retrain the model and include the validation set in the training data? If yes, how?
Or do I just use the model trained with early stopping and test it on the test set without retraining the model using the validation set?
