3
$\begingroup$

I know that there are similar questions about this topic, but my question is not really which metric should we depend on but why they are not equivalent?

There is a saying in the book Basic Econometrics

high zero-order correlations are a sufficient but not a necessary condition for the existence of multicollinearity

However, in the case of my project, I do detect the evidence of a high correlation coefficient and low VIF. May someone explains to me?

Thanks!

| cite | improve this question | |
$\endgroup$
3
$\begingroup$

What do you consider to be a high correlation coefficient? What do you consider to be a low VIF?

The VIF is calculated by regressing predictor $i$ on all the other predictors, and then calculating $VIF = \frac{1}{1 - R_i^2}$. If you consider a VIF of 5 to be high, you'd only get a high VIF if $R_i^2$ was greater than or equal to 0.8.

Now imagine that you have two predictors $i$, and $j$ that have a correlation coefficient of 0.8, which is fairly high, but no other predictors are correlated with predictor $i$ or $j$. When you then regress $i$ on all the other variables, $R_i^2$ will be slightly greater than the square of the correlation coefficient between $i$ and $j$, which is $0.8^2 = 0.64$ (smaller than the 0.8 needed to achieve a VIF of 5). In other words, the VIF's for $i$ and $j$ will be small, even though the correlation between them is high.

Basically the VIF for predictor $i$ captures how well all other predictors can explain predictor $i$. But to get a VIF that's considered high (greater than or equal to 5), there has to be a very strong fit when regressing predictor $i$ on the other variables. As demonstrated, it's definitely possible to have a "high" correlation between two variables, but still have "low" VIF's.

As far as your quote, the opposite can happen as well. You could have low pairwise correlations, but have high VIF's. This is because it's possible that there's a strong relationship between predictor $i$ and all the other variables together, even though there's not a high correlation between predictor $i$ any other predictor independently.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you! Since the correlation coefficient and VIF do not guarantee each other, can we say we should always refer to VIF as a fair measure of multicollinearity? For example, can we say a high correlation coefficient does not necessarily lead to multicollinearity? If so, I am wondering if xj and xi are highly correlated, while they are uncorrelated with the other variables (result in low VIF), shouldn't it still be hard to separate the impact of xi and xj on y and thus result in high standard deviation? $\endgroup$ – Yuan Jan 17 at 7:35
  • $\begingroup$ From what I've read, most people suggest to defer to the VIF for detecting multicollinearity rather than pairwise correlations. As this post suggests, high correlations may not always be problematic: stats.stackexchange.com/questions/262291/…. That being said, there's no standard for exactly which predictors to keep in the model. If the predictors have a high correlation and are connected, based on your intuition and knowledge about the process, it may be good to drop one, even if VIF's are low. $\endgroup$ – Nick Koprowicz Jan 17 at 14:47
  • $\begingroup$ Thanks Nick! There is still a point I am confused about: In your example, xi and xj are highly correlated, but the VIF is low. Wouldn't it still be difficult to disentangle the impact of xi and xj on y even though the VIF suggests a low multicollinearity degree. Is it a counterexample of the reliability of VIF as an indicator of multicollinearity. (However, the formula of sd of estimator includes VIF instead of correlation) $\endgroup$ – Yuan Jan 17 at 18:34
  • $\begingroup$ It could be difficult to disentangle the impact of xi and xj on y, but that doesn't mean VIF isn't reliable as an indicator of multicollinearity, since identifying and addressing multicollinearity is a somewhat subjective process. What I think it does show is that the many tools used to assess multicollinearity (correlations, VIF's, condition indices) can have different results, and so they should all be used, along with your knowledge of the problem, to decide which predictors to ultimately keep in your model. $\endgroup$ – Nick Koprowicz Jan 17 at 21:33
  • $\begingroup$ Thanks! I found such a formula that var(β_hat) = (σ^2/Σx^2)*VIF. So does it mean that even though the correlation is high, a variable with a low VIF will not be associated with the inflation of sd of estimator? $\endgroup$ – Yuan Jan 17 at 23:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.