0
$\begingroup$

For multiple regression, sometimes highly correlated independent variables do not exhibit multicollinearity measured by Variance Inflation Factor.

What could be the reason/reasons?

Can some one suggest a good reading material also?

$\endgroup$
4
1
$\begingroup$

What do you mean by "high" correlation?

$VIF = \frac{1}{1-R^2_j}$

If there are only 2 independent variables, then this is pretty simple. If the corrleation (r) is 0.8 then $VIF = \frac{1}{1-0.64} = \frac{1}{0.36} = 2.77$ Is 0.6 'high'?

But if there are more variables, then it gets trickier.

$\endgroup$
0
$\begingroup$

Consider the definition of the VIF: $$ \text{VIF} = \frac{1}{1-R^2_j} $$ Where $R^2_j$ is the coefficient of determination of predictor variable $x_j$. Now $$ R^2_j = 1 - \frac{\sum_i e_i}{\sum_i(y_i-\bar{y})^2} $$ which gets larger as the sum of the residuals (errors) in the numerator gets smaller. A larger $R^2_j$ will in turn cause the VIF to get larger. Geometrically speaking a line that fits snugly between the points of two predictor variables on a graph will have little error and thus a large VIF.

The Pearson correlation is different than fitting a line through data. It measures how well two variables are linearly correlated and does not attempt to find a line that best minimizes the error between it and the data points. The following plot is the best explanation.

VIF Plot

Here we see a strong correlation between the predictor variables $a$ and $b$ and a relatively low VIF. This is because it is difficult to fit a line through this data set but easy to see that the variables are well correlated.

$\endgroup$
5
  • $\begingroup$ These variables are not well correlated. You seem to pull a linguistic sleight-of-hand at the end by changing the sense of "correlated" from "having a high correlation coefficient" to "being clearly related." They are not the same! $\endgroup$
    – whuber
    Aug 24 '17 at 15:13
  • $\begingroup$ $-.6$ isn't bad. But let's say the correlation is somewhat low, I do think the example illustrates the idea. I'm not trying to be deceptive. The numbers are there for the reader to interpret regardless of the subjective description. $\endgroup$
    – Chris
    Aug 24 '17 at 15:19
  • $\begingroup$ Since OLS regression and VIF are entirely based on first and second order moments, you confuse the discussion by introducing data patterns that cannot be discriminated on the basis of these low moments. Your illustration is (at best) tangential to the real issues. $\endgroup$
    – whuber
    Aug 24 '17 at 15:22
  • $\begingroup$ I look forward to your elucidating answer :). $\endgroup$
    – Chris
    Aug 24 '17 at 15:23
  • 1
    $\begingroup$ I have posted many. For a few, please see the links I mention in a comment to the question. $\endgroup$
    – whuber
    Aug 24 '17 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.