3
$\begingroup$

I understand that variance inflation factors can be used to detect multicollinearity.

What is the intuition behind the VIF formulation? What aspect of this formula shows it detects multicollinearity between the predictors?

$\endgroup$

1 Answer 1

1
$\begingroup$

The VIF formula for predictor variable $k$ is defined as

$$ VIF_k = \frac{1}{1-R^2 _k }$$

where $R^2_k$ is the coefficient of multiple determination of the regression of the variable $k$ on all other predictor variables. If there is linear dependence among predictors the $R^2$ would then be high because this is precisely what it measures, the explained variance. But when $R^2$ is large the denominator will be small and consequenty the $VIF$ will be large. In fact, it will be unbounded for values of $R^2$ quiet close to $1$ - I believe $10$ is the number frequently used to indicate trouble but that is rather arbitrary.

Note that this is a better approach of diagnosing VIF than looking at pairwise correlation coefficients as there might be a linear relationship among more than two variables and this cannot be detected by the correlation coefficient. A better yet measure is the minimum eigenvalue of the predictor covariance matrix which as you know will be very close to zero in case of degeneracies.

$\endgroup$
2
  • $\begingroup$ "...where $R^2_k$ is the coefficient of multiple determination of the regression of the variable kk on all other predictor variables.." Does this mean that to get $R^2_k$, we compute a least squares model where variable k is the response variable and the other variables become the predictors $\endgroup$
    – Minaj
    Jan 13, 2016 at 23:45
  • $\begingroup$ @Minaj All other predictor variables, yes. This is precisely what that means. $\endgroup$
    – JohnK
    Jan 13, 2016 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.