For multiple regression, sometimes highly correlated independent variables do not exhibit multicollinearity measured by Variance Inflation Factor.
What could be the reason/reasons?
Can some one suggest a good reading material also?
$\begingroup$
$\endgroup$
4
-
$\begingroup$ How are you measuring the collinearity of the variables in question? $\endgroup$– ChrisCommented Aug 23, 2017 at 2:43
-
$\begingroup$ Using VIF. updated question. $\endgroup$– user2728024Commented Aug 23, 2017 at 5:46
-
1$\begingroup$ Inter alia, please see stats.stackexchange.com/questions/34488 and stats.stackexchange.com/a/14528/919 (for an explicit example of not-too-high VIFs in the presence of correlation problems in a multiple regression). $\endgroup$– whuber ♦Commented Aug 24, 2017 at 15:16
-
1$\begingroup$ Possible duplicate of Either quadratic or interaction term is significant in isolation, but neither are together $\endgroup$– kjetil b halvorsen ♦Commented Jun 17, 2019 at 9:47
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
What do you mean by "high" correlation?
$VIF = \frac{1}{1-R^2_j}$
If there are only 2 independent variables, then this is pretty simple. If the corrleation (r) is 0.8 then $VIF = \frac{1}{1-0.64} = \frac{1}{0.36} = 2.77$ Is 0.6 'high'?
But if there are more variables, then it gets trickier.
answered Jun 18, 2019 at 19:40
$\begingroup$
$\endgroup$
5
Consider the definition of the VIF: $$ \text{VIF} = \frac{1}{1-R^2_j} $$ Where $R^2_j$ is the coefficient of determination of predictor variable $x_j$. Now $$ R^2_j = 1 - \frac{\sum_i e_i}{\sum_i(y_i-\bar{y})^2} $$ which gets larger as the sum of the residuals (errors) in the numerator gets smaller. A larger $R^2_j$ will in turn cause the VIF to get larger. Geometrically speaking a line that fits snugly between the points of two predictor variables on a graph will have little error and thus a large VIF.
The Pearson correlation is different than fitting a line through data. It measures how well two variables are linearly correlated and does not attempt to find a line that best minimizes the error between it and the data points. The following plot is the best explanation.
Here we see a strong correlation between the predictor variables $a$ and $b$ and a relatively low VIF. This is because it is difficult to fit a line through this data set but easy to see that the variables are well correlated.
-
$\begingroup$ These variables are not well correlated. You seem to pull a linguistic sleight-of-hand at the end by changing the sense of "correlated" from "having a high correlation coefficient" to "being clearly related." They are not the same! $\endgroup$– whuber ♦Commented Aug 24, 2017 at 15:13
-
$\begingroup$ $-.6$ isn't bad. But let's say the correlation is somewhat low, I do think the example illustrates the idea. I'm not trying to be deceptive. The numbers are there for the reader to interpret regardless of the subjective description. $\endgroup$– ChrisCommented Aug 24, 2017 at 15:19
-
$\begingroup$ Since OLS regression and VIF are entirely based on first and second order moments, you confuse the discussion by introducing data patterns that cannot be discriminated on the basis of these low moments. Your illustration is (at best) tangential to the real issues. $\endgroup$– whuber ♦Commented Aug 24, 2017 at 15:22
-
$\begingroup$ I look forward to your elucidating answer :). $\endgroup$– ChrisCommented Aug 24, 2017 at 15:23
-
1$\begingroup$ I have posted many. For a few, please see the links I mention in a comment to the question. $\endgroup$– whuber ♦Commented Aug 24, 2017 at 15:25