5
$\begingroup$

I am reading the 《deep learning with python》. In chapter 4, about Fighting overfitting, I have two questions.

  1. why increasing epochs may cause overfitting? I know increasing will cause more gradient descent, will more gradient descent can cause overfitting?

  2. During the process of fighting overfitting, will the accuracy be reduced ?

Improve this question
$\endgroup$
1
  • $\begingroup$ It's not guaranteed that you overfit. However, typically you start with an overparameterised network ( too many hidden units), but initialised around zero so no effect.. increasing epochs will mean you fit better and better to training data. Checking the validation data allows you to see whether performance on unseen data is also improving ( on average). It's quite possible that performance for part of input space improves for training and validation,and for other parts improves only for training, gets worse for validation $\endgroup$
    – seanv507
    Commented Dec 27, 2018 at 9:48

1 Answer 1

Reset to default
5
$\begingroup$

Following is an intuitive explanation.

Any data = Behavior Pattern + Noise

Our objective is to find the pattern using Deep Learning model in the form of weights and biases for various nodes.

The objective of the DL model is to minimum error or maximum accuracy. In the process of minimizing error, it would have learnt the pattern but the error would not reach a absolute zero. If the training is continued for more epochs, model tries to reach zero error where it is starting to learn the noise of training data.

This noise can change from data sample to sample and hence you would observe the following curves when number of epochs is plotted against accuracy / error for training vs validation sets.

Yes. Accuracy would drop marginally but our objective was NOT to be good only on training data. Hence generalization / regularization is required for the model. Early Stopping is one technique that can help.

Number of Epochs vs Accuracy

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.