# Should a model be trained until it is stable to find optimal hyperparameters?

A model may take several days to train until it reaches an equilibrium - say if the change in error between epochs is lower than some threshold $$\epsilon$$, or accuracy reaches some equilibrium.

When tuning hyperparamters (via cross validation or otherwise), is there any problem in only training the model for a small number of epochs, so that training completes faster and more hyperparameters can be evaluated in a shorter amount of time? Training for a small number of epochs means the model hasn't reached an equilibrium as described above, but it has learned something so is performing far better than the random initialization.

• Sometimes the "plateau" is a bad place, and just a few steps down the gradient is both "good enough" in terms of quality and "desirable" in the sense of not over-fitting. – EngrStudent - Reinstate Monica Dec 4 '19 at 18:25