I'm trying to figure out the optimal number of epochs I need to train a convolutional network for, using 4 fold cross validation. I divided the dataset into 4 parts, trained the model on 3 parts, while keeping out one for validation. Repeated this thrice with the other parts as validation set.

After this experiment I found that, models on first two folds achieved 100% accuracy on validation set. The number of epochs it takes for each fold to achieve the highest possible accuracy on validation fold is wide apart viz. 21(100% on validation fold), 34(100%),36(98.97%) , 40(99.65%). My idea was to find the optimal number of epochs and then train the model on all the data for that many epochs. How do I choose the number of epochs from this experiment? If this approach is inadequate, how should I change it?


1 Answer 1


Think of accuracy on the validation set as an estimate of accuracy on future data, given the value of some hyperparameter. In this case, the hyperparameter of interest is the number of training epochs. So: for each CV fold, train the network (e.g. up to some maximum number of epochs). After each epoch, record accuracy on the validation set. Compute the average validation set accuracy (across CV folds) for each number of epochs. Choose the number of epochs that maximizes this value.

  • $\begingroup$ If the OP takes average of performances on different epochs (21, 34, 36, 40 as OP has mentioned), then the resulting average is misleading right? It can't be considered as a proper result, right? Epochs have to be same right? I'll be thankful if you can clarify this. $\endgroup$ Feb 6, 2021 at 6:26
  • 1
    $\begingroup$ @PrasanjitRath The goal of cross validation here is to estimate the expected generalization performance, given the number of epochs. So yes, when you average the performance across folds, it should always be the performance at the same number of epochs. $\endgroup$
    – user20160
    Feb 6, 2021 at 19:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.