# How do I test if two random variables are independent

I'm trying to understand the concept of independence experimentally using computer simulation.

I imagine to understand this concept:

1. I should be able to generate three data sets from random Variables A, B and C where A and B are independent and A and C are not independent.
2. I should be able to run some test where I compare data from A and B (and A and C) and the test gives me a measure of independence.

Could I get pointers of a resource (paper, textbook chapter, code, blog post, etc.) where there is sufficient information for me to experimentally understand the concept of independence.

• What statistical language do you use? – gung - Reinstate Monica Sep 15 '13 at 21:41
• @gung -- corrected. – user1172468 Sep 15 '13 at 21:42
• @gung, I'm kinda re-learning statistics -- I program using Java, matlab, lisp but would be open to learning R -- however I'd like to write this in a general purpose programming language such as Java – user1172468 Sep 15 '13 at 21:42
• Unless you restrict the form of dependence you're interested in, defining a single measure of dependence that has much power to detect the various forms is a pretty difficult task. For example, correlation is pretty good at finding linear dependence in the mean, but not much good at discovering the kind of dependence where the $(X,Y)$ values all lie near a circle. – Glen_b -Reinstate Monica Sep 15 '13 at 23:48

I don't know Java, so someone else will have to help you with the code. In addition, I would think any decent regression textbook would cover the idea of independent random variables from a basic applied perspective, if you wanted to know about it from a mathematical perspective, any introduction to mathematical statistics should cover that fairly early on.

In a sense, the issue is hard to nail down: If two variables are not independent, there are potentially infinite ways they could be dependent. For the sake of simplicity, let's specify the type of non-independence as being linearly correlated. I'm sure Java has functions that will do this for you, but you can get a sense of how to generate linearly correlated data from my answer here: How to generate correlated random numbers (given means, variances and degree of correlation)? Likewise, you could test for (this type of) non-independence with a simple product-moment correlation test. Of course, if you were interested in some more complicated, non-linear form of correlation, then we would need to specify that in order to determine how to generate and / or test it.

I do not recommend using correlations as a measure of independence. Basically Pearson product-moment correlation coefficient is a measure of linear correlation, which is often not the case. On the other hand Spearman's rank or Kendall tau rank correlations works also for non-linear correlated variables. However, my understanding about those procedures is that if those procedures might fail to "capture" the correlation, that does not mean at all that automatically those variables are independent.

More appropriate for this kind of task are what are called independence tests, and among them is the well-known Pearson's chi-squared test. See wiki page. I have no experience with using independence test, but the last book I studied states that is more appropriate to use G-Test. More details here on wiki page.

I currently build my own library for statistical and ml stuff and I did not yet implemented those tests, only some correlations. But I take a look on those pages and does not seem to be complex or hard to implement in any language.

I expect, however, to have some troubles generating random samples for distributions other than discrete uniform or normal Gaussian, since as far as I know, Java offers support only for those distributions.

• You're right that Pearson's $r$ will only work well for linear correlations, but the chi-squared & G-tests won't work well for continuous variables. Without specifying the form of the dependence, as @Glen_b notes above, there won't be a general way to do this. – gung - Reinstate Monica Sep 16 '13 at 1:00
• @gung I agree, I was thinking yesterday after I commit the message that those tests won't work well for continuous variables. However I do not know if deleting a response after publish is a way to go, so I will simply correct that – rapaio Sep 16 '13 at 7:24
• It's always OK to edit, or even delete, your posts, @Aurelian Tutuianu. You make some good points here though, so CV will be best off if you didn't delete this, but it's your choice. You could just add a caveat that this is a good strategy with categorical variables. (The truth is, I just assumed the OP meant continuous variables; nothing in the question states that.) – gung - Reinstate Monica Sep 16 '13 at 12:54