0
$\begingroup$

I have a conceptual question about why (processing power/storage aside) would you ever just use a regular linear regression without adding polynomial features? It seems like adding polynomial features (without overfitting) would always produce better results? I know linear regression can fit more than just a line but that is only once you decide to add polynomial features correct? My experience with python using sklearn's libraries.

$\endgroup$
  • $\begingroup$ It depends on your model and data. Do you have a prior on your data? Do you know that it is linear or not? Do you want to extrapolate? $\endgroup$ – Dorian Jun 24 at 16:59
  • 2
    $\begingroup$ Polynomial regression is extremely dangerous for extrapolation. If you only want interpolation then other methods such as splines or generalized additive models can provide more flexibility than simple polynomials. $\endgroup$ – Henry Jun 30 at 14:15
0
$\begingroup$

At a minimum, you should consider cross-posting this to the Data Science stack exchange site (stats is more in tune with the statistical, ie math, underpinnings).

That said, there is a trade-off between variance and bias in pretty much every model of this sort. Essentially, there are a few factors here.

  1. Overfitting. You dismissed this with a handwave but it's one of the most critical aspects. A high bias model (fewer parameters in this case) helps guard against overfitting -- but at the cost of sensitivity.
  2. There is a sweet spot in increasing the complexity of the model. At some point, it might not improve the fit to the training data.

Additionally, that hand-wave of setting "processing power/storage aside" really is non-trivial as are the labor-hours involved.

I hope that helps but, again, this (and a number of questions like it) are better asked here.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

There's a lot of nuance to this, as a general rule you chose the simpler model all things being equal. After building a polynomial model, you can F-test against a first-order model and go from there. Here is a link to more on this.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.