# Early stopping together with hyperparameter tuning in neural networks

Similar to this question (hyperparameter tuning in neural networks), I have a neural network with a similar list of parameters as the link above:

• Learning rate: $$[0.001, 0.01, 0.1]$$
• $$L_1$$ penalty: $$[0.01, 0.05, 0.1, 0.5]$$
• Early stopping tolerance: $$[0.0001, 0.001, 0.01]$$

The paper I'm replicating didn't use dropout, but they also didn't specify exactly how they've done hyperparameter tuning. So I've reserved a portion of data for choosing learning rate and L1 penalty, but for how many epochs do I train?

This is where early stopping comes in. I can either further split my training data and use a smaller portion just for early stopping purposes. Or I can use my larger validation set for early stopping and use the validation error for when training is stopped to also choose my hyperparameters. Conceptually, I would train my model solely in the training set and choose hyperparameters using the validation set, but having training stopped-early and choose hyperparameters at the same time seem to require the supposedly "unseen" data during training. Which method should I use?