0
$\begingroup$

I'm pritty new to the machine learning world, and I ws trying to figure out how many epochs should I run my training CNN model on the MNIST dataset (which has 60,000 training images and 10,000 validation images).

Now, if I undestand correctly, an epoch is defined such:

how many times I train my model with the WHOLE dataset

and I have seen many tutorial blogs showing that they ran 25 epochs on the whole MNIST dataset.

isn't it causing overfitting? shouldn't the model train on the whole data only once? I would appreciate an explanation on it, because there are many other parameters such as "steps_per_epoch", which I dont know how to set becuase I can't figure out what should I expect when I train a model (like, when can I tell that i'm doing way too many epochs and such).

$\endgroup$
1
  • 1
    $\begingroup$ Usually you'd monitor the loss on a validation dataset. (see here for an explanation and example.) $\endgroup$
    – Ben
    Jul 22, 2021 at 14:04

1 Answer 1

4
$\begingroup$

When you train a neural network using stochastic gradient descent or a similar method, the training method involves taking small steps in the direction of a better fit. Each step is based on one minibatch of data, and an epoch means you have made one step based on every data point. But that's only one small step!

Typically, you need to take many more steps than just one based on each data point in order to get a good fit. It's hard to predict in advance how many steps and how many epochs will be needed -- partly, it's hard to tell how close to the minimum you are, and partly, you may well not want to go all the way to the minimum because of overfitting. It is common to monitor the goodness of fit on test data as you train, so that you can stop when the test-data accuracy stops decreasing or starts increasing.

$\endgroup$
3
  • $\begingroup$ ok, so based on what u have said (which was helpful, thank you), would it be smart to split the data into many epoch? for example, if MNIST has 60,000 train images, I could split it into 6 epochs of 10,000 different images. would it be smarter than doing lots of epochs on the whole data? $\endgroup$ Jul 17, 2021 at 3:48
  • $\begingroup$ No, there's no advantage of that as far as I know. $\endgroup$ Jul 17, 2021 at 5:15
  • $\begingroup$ This is a great answer; I also wanted to highlight the role of early stopping in helping avoid overfitting. It's often used in practice and implemented in most major deep learning libraries. $\endgroup$ Jul 17, 2021 at 7:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.