5
$\begingroup$

I have a Box-Cox regression where the explanatory variables are almost all dummy variables. If I want to see if there is multicollinearity among them, what would be an appropriate test? Do variance inflation factor (VIF) tests work here? What about the Pearson correlation coefficient matrix?

$\endgroup$
  • $\begingroup$ In addition to Mike's answer, this post might also be of interest. $\endgroup$ – COOLSerdash Aug 13 '13 at 18:56
3
$\begingroup$

The VIF is probably the best way to go here. The Pearson correlation will give you a lousy measure here because it behaves somewhat weirdly for categorical variables like this. Another possibility is to use a matrix of a different measure like cosine similarity: $\sum x_i*x_j / \sqrt{\sum x_i^2 * \sum x_j^2}$. I think that is equivalent to Spearman's Rho or Kendall's Tau but am not sure off the top of my head.

I'd stick to the VIF though because it will tell you for each variable whether the other variables combined are highly colinear. But if you want a visual diagnostic of which pairwise variables are similar, those other metrics are better than Pearson for categorical data.

----EDIT---

Sure. This has to do primarily with the fact that Pearson's correlation can swing up or down or go negative very easily. Here's an example:

> cor(c(0,1,1,1,0,1,0,1,0),c(1,1,0,1,1,0,1,1,0))
[1] -0.1581139
> cor(c(0,1,1,1,0,1,0,1,0),c(0,1,0,1,1,0,1,1,0))
[1] 0.1

Here, by changing just one of the entries to zero we have swung the correlation from positive to negative. But the VIF uses $1/(1-R_{i}^2)$ where the $R_{i}^2$ is for the regression of the other variables on the one in question. I would have to work it out but I think that is basically a linear combination of something similar to the cosine measure I posted above, or a transform of it. Essentially though, it can't go negative.

I don't know any literature on it off the top of my head, but I will think about it.

$\endgroup$
  • $\begingroup$ Thanks Mike. Could you please explain some more about why VIF is still appropriate for dummy variables, when Pearson is not? Is there a publication reference that you could point me to which discusses this? I have done an extensive search online and found nothing for this topic, which is quite surprising since I am sure others before me have encountered this problem, which is quite common, I would think. You're the first person who has specifically said this, so thanks for your reply and hope to hear from you soon!!! $\endgroup$ – kellyyang Aug 15 '13 at 14:55
  • $\begingroup$ I am not sure what your example means, could you pls explain it in words? Also, why is VIF appropriate? Is there any publication references you can give me? Many many thanks! $\endgroup$ – kellyyang Aug 16 '13 at 15:43
  • $\begingroup$ Sorry about that, I had started to put it in a comment, then decided to add it to the post by editing but didn't realize it had saved half the comment. I hope that's a little more clear but let me know if it's not. $\endgroup$ – Mike Nute Aug 16 '13 at 22:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.