0
$\begingroup$

While doing multiple regression, some of my predictors are correlated. But it's not collinear. The results are given below.

y - dependent variable a,b,c - independent variables

Correlation

          y           a          b             c
y         1.0000000   0.5418774  0.5047409     0.4394508
b         0.5418774   1.0000000  0.8532455     0.7017283
c         0.5047409   0.8532455  1.0000000     0.6983398
d         0.4394508   0.7017283  0.6983398     1.0000000

vif

 a             b             c 
 3.995067      3.965175      2.127463 

Can I include both the variables in the model? After including both the variables in the model I am getting decent results and p value for coefficients for both variables are also significant.

Residuals:
     Min       1Q   Median       3Q      Max 
-13.3627  -1.3381   0.1509   1.4823   9.1753 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)    7.62649    0.02669  285.79   <2e-16 ***
a              0.78397    0.01473   53.23   <2e-16 ***
b              0.31077    0.01899   16.36   <2e-16 ***
c              0.21349    0.01214   17.59   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 2.133 on 54635 degrees of freedom
Multiple R-squared:  0.3048,    Adjusted R-squared:  0.3048 
F-statistic:  7985 on 3 and 54635 DF,  p-value: < 2.2e-16

I read Collinearity implies correlation but not vice- versa, so is it okay to include correlated variables in a linear model if they are not collinear?

$\endgroup$
  • $\begingroup$ What is it you want to learn from, or do with, this model? $\endgroup$ – gung Aug 4 '17 at 18:52
  • $\begingroup$ I want to identify the coefficients for each variable from this model. $\endgroup$ – user2728024 Aug 4 '17 at 18:55
  • $\begingroup$ You have the fitted coefficients, so you're fine. The CIs for the coefficients will be approximately 2X wider than they would have been ($\sqrt{\approx 4} \approx 2$) if the VIFs had been 1. So you have less precision in your estimates. If you compute the CIs and incorporate them into your interpretations, you are fine. $\endgroup$ – gung Aug 4 '17 at 19:03
  • $\begingroup$ Yeah. I just wanted to confirm that the correlation doesn't influence my the model. Thanks for the reply. $\endgroup$ – user2728024 Aug 4 '17 at 19:31
1
$\begingroup$

Yes, it is okay to include correlated variables. The Variance Inflation Factors (VIF) only works as an index that shows you how much of the variance of the estimators came from the collinearity.

Just bear in mind that if the VIFs are too high, that might compromise your inference. Some textbooks say that you should only be worried if your VIFs are larger than 10.

$\endgroup$
  • $\begingroup$ Thank you. I am considering 4 as the threshold. For the above-mentioned model, some variables have .85 correlation and it is okay in the model? $\endgroup$ – user2728024 Aug 4 '17 at 18:35
  • $\begingroup$ What really matter is the VIF, not the correlation between variables itself. If your VIF is below 4 and you still find significant p-values, I would not worry. $\endgroup$ – Gustavo Amarante Aug 4 '17 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.