Say I'm training a deep neural network and I have split my data into train, val, and test splits. I have trained many models on the train data and then using the val data during the training loop for things like ReduceLRonPlateau and early stopping. Then I test all my models on the test set and determine that one of my models performs the best. Then, I would like to use the hyperparameters I used in my best model to train a production model and I would like to use as much of the data as I can. I would like to understand the best practices around doing this.
One option would be to combine all the data into one set and train it using the learning rate schedule that my previous ReduceLRonPlateau produced and stopping it at the same number of epochs. But since I have more data this time, I would think a few extra epochs might help. Is this a bad idea?
Another option would be to combine the train and test sets and then use the val set again for ReduceLRonPlateau and early stopping. This way the training process is still based on my dataset. Is this a good idea? Is there a risk that my result would be worse than my model that was only based on the train split? Are there better options out there?