1
$\begingroup$

I was going through an example of Polynomial regression and could not understand why adding quadratic term doesn't violate Linear Model assumption of multi-collinearity as we are just squaring the term- that is, violation of variables not correlated.

Ex: mpg = β0 + β1 × horsepower + β2 × horsepower^2 +e

We are just squaring one variable to get the second variable.

Also, how would you interpret b1_hat and b2_hat?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$
  1. It is not an assumption that covariates in a linear model are independent$^1$
  2. If the covariates are symmetric and centered, the dot product of the linear and quadratic term would be 0, not that it matters. Otherwise, yes they are likely to have some non-zero covariance.
  3. In the presence of a quadratic term the coefficient to the linear term is the instantaneous rate of change of the quadratic trendline at $X=0$. The quadratic coefficient is interpreted like an interaction between a variable and itself. It is the difference of the difference in mpg for a unit difference in horsepower at a one-higher horsepower.

$^1$: In experimental design, we prefer to balance blocking factors this way, so that the data are suitable to answering a large variety of questions, especially if there is incidental evidence of interaction. The only reason that balance of covariates might matter in a prespecified regression model is that linear models fit with any combinations of the covariates will generally have the same estimated coefficient for each term. But this artifact of design is not a necessary condition for fitting or interpreting a linear model.

Improve this answer
$\endgroup$
3
  • $\begingroup$ hmm..regarding point 1. According to Wikipedia, Lack of Perfect Multi-collinearity is one of the assumption en.wikipedia.org/wiki/Linear_regression. And doesn't multi-collinearity mean "Multicollinearity refers to a situation in which two or more explanatory variables in a multiple regression model are highly linearly related. We have perfect multi-collinearity if, for example as in the equation above, the correlation between two independent variables is equal to 1 or −1. " [Wiki]. In our case, the correlation between our variables will be almost 1 no? $\endgroup$
    – leviathan
    Commented Oct 23, 2019 at 17:22
  • $\begingroup$ In our case, I think we treat horsepower and horsepower^2 as two variables. $\endgroup$
    – leviathan
    Commented Oct 23, 2019 at 17:31
  • 2
    $\begingroup$ @leviathan the distinction between "perfect" multi-collinearity and almost perfect is critical. If there is an exact linear dependency among the covariates (correlation between two of them at exactly 1 or -1 is only one type of example) then there is no unique solution to a least-squares model. That's why "Lack of Perfect Multi-collinearity" is a stated assumption. Otherwise there is a unique solution. As this answer notes in part 2, the level of the nominal correlation will differ depending on centering of the original covariate, but it won't be exactly +1 or -1. $\endgroup$
    – EdM
    Commented Oct 23, 2019 at 19:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.