# Multicollinearity and correlation in multiple regression

For multiple regression, sometimes highly correlated independent variables do not exhibit multicollinearity measured by Variance Inflation Factor.

What could be the reason/reasons?

Can some one suggest a good reading material also?

• How are you measuring the collinearity of the variables in question? Aug 23 '17 at 2:43
• Using VIF. updated question. Aug 23 '17 at 5:46
• Inter alia, please see stats.stackexchange.com/questions/34488 and stats.stackexchange.com/a/14528/919 (for an explicit example of not-too-high VIFs in the presence of correlation problems in a multiple regression).
– whuber
Aug 24 '17 at 15:16
• Jun 17 '19 at 9:47

What do you mean by "high" correlation?

$$VIF = \frac{1}{1-R^2_j}$$

If there are only 2 independent variables, then this is pretty simple. If the corrleation (r) is 0.8 then $$VIF = \frac{1}{1-0.64} = \frac{1}{0.36} = 2.77$$ Is 0.6 'high'?

But if there are more variables, then it gets trickier.

Consider the definition of the VIF: $$\text{VIF} = \frac{1}{1-R^2_j}$$ Where $R^2_j$ is the coefficient of determination of predictor variable $x_j$. Now $$R^2_j = 1 - \frac{\sum_i e_i}{\sum_i(y_i-\bar{y})^2}$$ which gets larger as the sum of the residuals (errors) in the numerator gets smaller. A larger $R^2_j$ will in turn cause the VIF to get larger. Geometrically speaking a line that fits snugly between the points of two predictor variables on a graph will have little error and thus a large VIF.

The Pearson correlation is different than fitting a line through data. It measures how well two variables are linearly correlated and does not attempt to find a line that best minimizes the error between it and the data points. The following plot is the best explanation.

Here we see a strong correlation between the predictor variables $a$ and $b$ and a relatively low VIF. This is because it is difficult to fit a line through this data set but easy to see that the variables are well correlated.

• These variables are not well correlated. You seem to pull a linguistic sleight-of-hand at the end by changing the sense of "correlated" from "having a high correlation coefficient" to "being clearly related." They are not the same!
– whuber
Aug 24 '17 at 15:13
• $-.6$ isn't bad. But let's say the correlation is somewhat low, I do think the example illustrates the idea. I'm not trying to be deceptive. The numbers are there for the reader to interpret regardless of the subjective description. Aug 24 '17 at 15:19
• Since OLS regression and VIF are entirely based on first and second order moments, you confuse the discussion by introducing data patterns that cannot be discriminated on the basis of these low moments. Your illustration is (at best) tangential to the real issues.
– whuber
Aug 24 '17 at 15:22
• I look forward to your elucidating answer :). Aug 24 '17 at 15:23
• I have posted many. For a few, please see the links I mention in a comment to the question.
– whuber
Aug 24 '17 at 15:25