0
$\begingroup$

I have a regression with 6 independent variables and one of them is highly negatively correlated with one of them. (-0.81 correlation) I was wondering if the regression is still valid or if it's biased? Are the variables still orthogonal? The goal of the regression is inference and all variables are important.

$\endgroup$
6
  • 3
    $\begingroup$ What is the goal of your regression, prediction or inference? And how interested are you in the doing inference on the two correlated variables? $\endgroup$ – Dave Jul 28 '20 at 18:47
  • $\begingroup$ You could try to use Ridge and Lasso to see which one of these variables are retained in the model with increasing regularity. You can also try bootstrapping/cross validation to assess the stability of the model. $\endgroup$ – Atakan Jul 28 '20 at 19:04
  • 1
    $\begingroup$ (1) The variables never were orthogonal. (2) Correlation does not introduce bias in linear regression. (3) A single high correlation coefficient is rarely of concern of itself. Whether there may be a multicollinearity problem is a matter of analyzing the full six-dimensional distribution of the independent variables. $\endgroup$ – whuber Jul 28 '20 at 19:10
  • $\begingroup$ Try to model one of the two predictors separately and then together. Do you see any substantial difference or inconsistency in their coefficients? If you do Bayesian regression with correlated variables, you may still end up with a stable coefficient, but one of the two may represent the effect conditional on the other variable, so be careful with the inference. statmodeling.stat.columbia.edu/2019/07/07/… $\endgroup$ – user289381 Jul 28 '20 at 19:13
  • 5
    $\begingroup$ Does this answer your question? How to deal with high correlation among predictors in multiple regression? $\endgroup$ – Durden Jul 30 '20 at 2:40
0
$\begingroup$

Correlation among regressors is perfectly fine, as long as it isn't perfect correlation (i.e., perfect multicollinearity). A correlation of -.81 is no reason for concern. The variance-inflation factor in this case would be :$$VIF = \frac{1}{1-(-0.81)^{2}} \approx 2.9$$ so clearly well below the usual rule-of-thumb cut-off value of 10.

Also, since you've mentioned it: regressors do not need to be orthogonal in linear regression for inference to be valid. You can make them orthogonal (using the Frisch-Waugh theorem), but you don't have to.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.