1
$\begingroup$

The four assumptions for bivariate regression are:
    • (L)inearity
    • (I)ndepdent observations
    • (N)ormal errors
    • (E)qual variance
And for multiple regression we add a fifth assumption:
    • no multicollinearity between predictors.

From a purely mathematical perspective, true multicollinearity makes the problem unsolvable in the sense that you cannot obtain a unique solution. But to be clear, this means there is perfect correlation between two or more predictors (or some subset of predictors lie on a smaller dimensional space). This, of course, is bad.

But, in practice, the assessment of this assumption results in the removal of nearly multi-collinear predictors. To clarify, I understand one pragmatic reason for this: even if your predictor variables are not perfectly correlated, if they are very highly correlated, the resulting design matrix may be computationally indefinite (i.e., not truly indefinite, but not solvable with machine number computational limitations). This, too, is bad.

Unfortunately, it seems to me that many textbook authors (particularly in the social sciences) have morphed the fifth assumption to "no NEARLY multi-collinear predictors". (E.g., remove items if the VIF is 10 or more...even though, the predictor variables may not be perfectly multi-collinear.)

My question is this: ¿is this "revised" assumption a valid assumption? or ¿is this a hopeful imposition for convenience's sake?

Note: This query is motivated for pedagogic purposes for introductory level statistics with applied focus (e.g., an applied statistics course for the social sciences).

Improve this question
$\endgroup$
8
  • 3
    $\begingroup$ Who adds that fifth assumption? It's unnecessary and doesn't appear in many good textbooks on multiple regression. $\endgroup$
    – whuber ♦
    Commented Mar 19, 2022 at 23:58
  • $\begingroup$ The revised assumption is nonsense. An assumption of no perfect multicollinearity is not. $\endgroup$
    – Richard Hardy
    Commented Mar 20, 2022 at 8:00
  • 2
    $\begingroup$ A more correct assumption is that the design matrix is full rank. It's possible to have a non-full-rank design matrix without perfect multicollinearity (e.g., including all levels of a factor as dummies when an intercept is present). Also, normality of errors is not required for unbiasedness of the OLS coefficients; it's only required for the t-statistics of the coefficients computed using the usual standard error to have a t-distribution under the null. $\endgroup$
    – Noah
    Commented Mar 20, 2022 at 8:20
  • $\begingroup$ That is not a standard assumption, but if it bothers you, do a PCA and take all of the principal components. Then you retain all of the information in all of the original variables, but you have uncorrelated features. While I am sympathetic to the idea that we might be able to capture almost as much information in the predictors by dropping one that is correlated with others, using fewer parameters and reducing the risk of overfitting, that seems not to pan out in practice, as I discuss here and link to additional reading. $\endgroup$
    – Dave
    Commented Mar 21, 2022 at 0:50
  • $\begingroup$ Some similar posts: stats.stackexchange.com/questions/220214/…, stats.stackexchange.com/questions/289424/…, $\endgroup$
    – kjetil b halvorsen ♦
    Commented Mar 21, 2022 at 1:25

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.