# How do I study the “correlation” between a continuous variable and a categorical variable?

What's a meaningful "correlation" measure to study the relation between the such two types of variables?

In R, how to do it?

Thanks a lot!

-
before you ask "how do you study", you should have the answer to "how do you define" :-) BTW, if you project the categorical variable to integer numbers, you can do correlation already. –  Tomas May 30 '12 at 16:40
@Tomas, if you do that, the estimated strength of the relationship depends on how you've decided to label the points, which is kind of scary :) –  Macro May 30 '12 at 16:47
@Macro, you are right - another solid argument for having a good definition! –  Tomas May 30 '12 at 16:50

For a moment, let's ignore the continuous/discrete issue. Basically correlation measures the strength of the linear relationship between variables, and you seem to be asking for an alternative way to measure the strength of the relationship. You might be interested in looking at some ideas from information theory. Specifically I think you might want to look at mutual information. Mutual information essentially gives you a way to quantify how much knowing the state of one variable tells you about the other variable. I actually think this definition is closer to what most people mean when they think about correlation.

For two discrete variables X and Y, the calculation is as follows: $$I(X;Y) = \sum_{y \in Y} \sum_{x \in X} p(x,y) \log{ \left(\frac{p(x,y)}{p(x)\,p(y)} \right) }$$

For two continuous variables we integrate rather than taking the sum: $$I(X;Y) = \int_Y \int_X p(x,y) \log{ \left(\frac{p(x,y)}{p(x)\,p(y)} \right) } \; dx \,dy$$

Your particular use-case is for one discrete and one continuous. Rather than integrating over a sum or summing over an integral, I imagine it would be easier to convert one of the variables into the other type. A typical way to do that would be to discretize your continuous variable into discrete bins.

There are a number of ways to discretzie data (e.g. equal intervals), and I believe the entropy package should be helpful for the MI calculations if you want to use R.

-
Thanks. But how high an MI is corresponding to the corr=1 and how low an MI corresponds to corr=0? –  Luna May 31 '12 at 17:58
MI has a minimum of 0, and MI = 0 if and only if the variables are independent. MI has no constant upper-bound though (the upper-bound is related to the entropies of the variables), so you might want to look at one of the normalized versions if that is important to you. –  Michael McGowan May 31 '12 at 19:41