0
$\begingroup$

I just started reading Introduction to Statistical Learning with R and I am currently trying to work through the exercises.

One of the questions is "What are the advantages and disadvantages of a very flexible (versus a less flexible) approach for regression or classification?"

when looking through online answers to check my own, I got right the advantages however I found this for the disadvantages:

The disadvantages for a very flexible approach for regression or classification are requires estimating a greater number of parameters, follow the noise too closely (overfit), increasing variance.

Though I understand the noise and variance disadvantages, I don't see the connection with parameters? How does the relation between parameters and Flexible models differ from the relation between parameters and non-flexible models?

$\endgroup$
0
$\begingroup$

It has to do with degrees of freedom in your equation. This is the phenomenon called the curse of dimensionality, and your book is trying to describe to you the incremental effect is has as the number of features or coefficients expands.

The higher dimensional your feature space, the more "thinly spread" your sample is across it, making it easier for the model to overfit to a more complex pattern, or more refined signal.

Worth pointing out that this relies on the new features have non-uniform distributions, aka the entire sample is not the same across that feature.

Hope that's helpful

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.