2
$\begingroup$

The difference between Linear regression and Polynomial regression is that in the last we manipulate our original explanatory variables in a way to create polynomial dependency between Y and X. For the sake of simplicity if we consider one feature (explanatory variable) only: X, the Polynomial regression with degree 2 will look like this:

$Y$ = $\beta_0$ + $\beta_1\cdot X$ + $\beta_2\cdot X^2$

In sklearn we can do this by initializing and calling fit_transform method of PolynomialFeatures(2). Eventually if we want to train our model we should use LinearRegression of sklearn to do so.

However, one of the assumptions of the Linear Regression is that all features should be uncorrelated. $X$ and $X^2$, on the other hand are perfectly correlated. So, doesn't this create any issue with the training process?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ There is no such assumption about uncorrelated predictors in linear regression. The Gauss-Markov theorem requires uncorrelated errors. Also, $X$ and $X^2$ are not perfectly correlated. $\endgroup$
    – Dave
    Commented Aug 11, 2020 at 18:13
  • 2
    $\begingroup$ My post at stats.stackexchange.com/a/384636/919 illustrates and explains the possibilities for multicollinearity when introducing a quadratic term in a regression. Additional insight can be had by searching our site for posts on orthogonal polynomials: the idea is that the very same model can be expressed in terms of two new variables $Z_1$ and $Z_2,$ derived as linear combinations of $X$ and $X^2,$ where $Z_1$ and $Z_2$ are centered and orthogonal: that is, there's no collinearity whatsoever. $\endgroup$
    – whuber ♦
    Commented Aug 11, 2020 at 18:21

0

Reset to default