3
$\begingroup$

This line is written in ISLR : Overfitting refers specifically to the case in which a less flexible model would have yielded a smaller test MSE.

I am unable to get this line, can anyone explain to me so that I can understand it.

As we know the flexibility of the model increases, the testMSE first decreases and then increases. In the case of underfitting, our model is less flexible and the test MSE would be smaller. But in the above line, it is referring a less flexible model to overfitting case!.

Can anyone explain, what I am interpreting wrong from the above line ?

$\endgroup$

2 Answers 2

5
$\begingroup$

Let's start out with some model A. On the training data, we get an MSE of 9. On the test data, we get an MSE of 10. Model A has 5 parameters.

Now we evaluate model B, which has 500 parameters. We can model more complicated behavior with this model than we could have with model A, meaning that model B is more flexible than A. When we look at our performance metrics, we find model B to have an MSE of 0 on the training data but 25 on the test data. Model B has overfit, agreed?

The less flexible model gave a smaller error on the test data.

That's what ISLR means.

$\endgroup$
1
  • 1
    $\begingroup$ Thanks for explaining. I was understanding that line as Overfiting means...this this. I understood the concept correctly at that time but not line. Now I got it. Thanks $\endgroup$ Aug 16, 2019 at 16:57
2
$\begingroup$

Great question and answer above.

The one thing I would add would be the importance of the test set in order to evaluate overfitting.

As Dave alluded to with his example, any model can essentially drive the training error/MSE/etc to 0 if you add enough parameters. Therefore, the key to assessing whether overfitting is occurring is evaluating whether the model generalizes to data it has never seen.

Under ideal situations, this would be a completely independent dataset (i.e., training model with data from one hospital and comparing it to a population from a different hospital). However, if you do not have this readily available, then you do this by splitting your data into training, validation, and test set via a sampling scheme of your choice.

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.