2
$\begingroup$

I am running a regression analyis in r:

fit <- lm(Cost ~ Slope + YardDist, data = test)

I want to test the two independent variables for multicollinearity. I tested it with vif() (from the car package) and kappa().

> vif(fit)
   Slope YardDist 
1.000121 1.000121 
> kappa(fit)
[1] 11631.87

VIF tells me there is no multicollinearity and kappa tells me there is very high multicollinearity. What is the difference between both and which one is 'right'?

$\endgroup$
5
$\begingroup$

If you only have two variables, you can just check the correlation between them. The VIF is:
$$ \text{VIF}=\frac{1}{\text{tolerance}}=\frac{1}{1-r^2} $$ On the other hand, kappa, is the condition number; that is: $$ \sqrt{\frac{\text{max(eigenvalue(X'X))}}{\text{min(eigenvalue(X'X))}}} $$ One thing that is often recommended with kappa is to center your variables first (note that there is difference of opinion about this recommendation). If your variables are far from 0, the sampling distributions of the $\beta_j$s will be correlated with the sampling distribution of $\beta_0$ (i.e., the intercept). I suspect that's what you are seeing here.

It might help you to read my question here: Is there a reason to prefer a specific measure of multicollinearity?

$\endgroup$
  • $\begingroup$ What does it mean 'to center the variables'? $\endgroup$ – ustroetz Jun 29 '13 at 3:40
  • 1
    $\begingroup$ @user1738154, to center a variable you subtract the mean of the variable from each value. Thus the mean of the new variable is 0, but the SD is the same as before. $\endgroup$ – gung Jun 29 '13 at 3:45
  • $\begingroup$ Wow, after centering my variables, the kappa value equals almost the VIF value. Thanks for pointing that out to me! $\endgroup$ – ustroetz Jun 30 '13 at 18:41
  • $\begingroup$ @user1738154, the upshot is that your SE / 95% CI for the intercept will be inflated / you will have lower power to differentiate your estimated intercept, $\hat\beta_0$, from $0$. This is due to the fact that your $X_j$ values are far from $x_j=0$ & there is some uncertainty about the slopes. However, most people aren't terribly concerned about this. If, OTOH, you are, you will need to run another study where your $X_j$s are centered on $x_j=0$. $\endgroup$ – gung Jun 30 '13 at 18:52
  • $\begingroup$ well my kappa value is 1.029708. Doesn't that mean there is no multicollinearity? For the regression function itself (which I am mainly interested in) I can use the non-centered variables, can't I? $\endgroup$ – ustroetz Jun 30 '13 at 19:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.