2
$\begingroup$

What would you go for in assessing collinearity - correlation or VIFs?

Say you run pairplots and calculate Pearson correlation coefficients between pairs of explanatory variables. Two of them have a correlation coefficient of around 0.8, which is rather high. This would suggest that including both variables in the same regression model might not be a good idea. But say you include them anyway. You then run the vif command from the car package. The Variance Inflation Factors (VIFs) are all below a threshold value (say 3, for example). This indicates no problematic collinearity.

Would you say that:

1) Both can be included in the model if VIFs are low, regardless of high correlation?

or

2) High correlation means one variable should be dropped immediately at the beginning, regardless of VIFs?

(This is not related to any particular data or model, hence no specific data example provided, but I have experienced this in the past. I know there have been threads related to this question, but I have not found an actual answer to my specific question.)

$\endgroup$
4
$\begingroup$

I would use condition indexes rather than either VIFs or correlations; I wrote my dissertation about this, but you can also see the work of David Belsley, e.g. this book. But if I had to choose between VIFs and correlations, I'd go with VIFs. Belsley shows that fairly high correlations are not always problematic.

If you are using R, another method that seems good to me is to use the perturb package to see if the collinearity is problematic.

$\endgroup$
  • 1
    $\begingroup$ (+1) Just to add that if in the practical use case the two variables have a substantive connection it might be appropriate to drop one. So for instance including two measures of depression may not be necessary. $\endgroup$ – mdewey Feb 16 '17 at 16:10
  • $\begingroup$ Thank you both. In the case that I had a while ago, I had two measured variables, length and width of a plant. But, these two are likely to be highly correlated, and indeed they were (around 0.8 or so). But the VIFs in the model showed apparently no problem. So in such a case, you would stick with the VIFs and keep them both in the model (assuming correlation and VIFs are the two options)? $\endgroup$ – Tilen Feb 16 '17 at 17:21
  • $\begingroup$ @mdewey, by substantive connection did you mean correlation, or connection as judged otherwise, for example by reason and using your general knowledge of the study system? $\endgroup$ – Tilen Feb 17 '17 at 9:36
  • 1
    $\begingroup$ I agree with @mdewey . And you decide in practice by thinking about it. Sorry. I don't mean to be rude here and am not implying that you aren't thinking. But statistics can't answer that question. Do you want to control for the other variable or not? In your case, do you want to look at effect of length after controlling for width or not controlling for width? $\endgroup$ – Peter Flom - Reinstate Monica Feb 18 '17 at 11:32
  • 1
    $\begingroup$ @PeterFlom no worries, it wasn't perceived as rude. I agree with you completely, and I am indeed thinking about it, which led me to write this question. :) In my example, length and width were highly correlated, so I thought it would be prudent to drop one. But then VIFs did not indicate any problem, which left me confused. With respect for controlling or not controlling, I only want to see whether any of these two variables seem to affect the response, and in which way. $\endgroup$ – Tilen Feb 20 '17 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.