3
$\begingroup$

I am running a regression analyis in r:

fit <- lm(Cost ~ Slope + YardDist, data = test)

I want to test the two independent variables for multicollinearity. I tested it with vif() (from the car package) and kappa().

> vif(fit)
   Slope YardDist 
1.000121 1.000121 
> kappa(fit)
[1] 11631.87

VIF tells me there is no multicollinearity and kappa tells me there is very high multicollinearity. What is the difference between both and which one is 'right'?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

If you only have two variables, you can just check the correlation between them. The VIF is:
$$ \text{VIF}=\frac{1}{\text{tolerance}}=\frac{1}{1-r^2} $$ On the other hand, kappa, is the condition number; that is: $$ \sqrt{\frac{\text{max(eigenvalue(X'X))}}{\text{min(eigenvalue(X'X))}}} $$ One thing that is often recommended with kappa is to center your variables first (note that there is difference of opinion about this recommendation). If your variables are far from 0, the sampling distributions of the $\beta_j$s will be correlated with the sampling distribution of $\beta_0$ (i.e., the intercept). I suspect that's what you are seeing here.

It might help you to read my question here: Is there a reason to prefer a specific measure of multicollinearity?

Improve this answer
$\endgroup$
8
  • $\begingroup$ What does it mean 'to center the variables'? $\endgroup$
    – ustroetz
    Commented Jun 29, 2013 at 3:40
  • 1
    $\begingroup$ @user1738154, to center a variable you subtract the mean of the variable from each value. Thus the mean of the new variable is 0, but the SD is the same as before. $\endgroup$
    – gung - Reinstate Monica
    Commented Jun 29, 2013 at 3:45
  • $\begingroup$ Wow, after centering my variables, the kappa value equals almost the VIF value. Thanks for pointing that out to me! $\endgroup$
    – ustroetz
    Commented Jun 30, 2013 at 18:41
  • $\begingroup$ @user1738154, the upshot is that your SE / 95% CI for the intercept will be inflated / you will have lower power to differentiate your estimated intercept, $\hat\beta_0$, from $0$. This is due to the fact that your $X_j$ values are far from $x_j=0$ & there is some uncertainty about the slopes. However, most people aren't terribly concerned about this. If, OTOH, you are, you will need to run another study where your $X_j$s are centered on $x_j=0$. $\endgroup$
    – gung - Reinstate Monica
    Commented Jun 30, 2013 at 18:52
  • $\begingroup$ well my kappa value is 1.029708. Doesn't that mean there is no multicollinearity? For the regression function itself (which I am mainly interested in) I can use the non-centered variables, can't I? $\endgroup$
    – ustroetz
    Commented Jun 30, 2013 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.