I'm trying to figure out the optimal number of epochs I need to train a convolutional network for, using 4 fold cross validation. I divided the dataset into 4 parts, trained the model on 3 parts, while keeping out one for validation. Repeated this thrice with the other parts as validation set.
After this experiment I found that, models on first two folds achieved 100% accuracy on validation set. The number of epochs it takes for each fold to achieve the highest possible accuracy on validation fold is wide apart viz. 21(100% on validation fold), 34(100%),36(98.97%) , 40(99.65%). My idea was to find the optimal number of epochs and then train the model on all the data for that many epochs. How do I choose the number of epochs from this experiment? If this approach is inadequate, how should I change it?