3
$\begingroup$

I'm trying to understand the concept of independence experimentally using computer simulation.

I imagine to understand this concept:

  1. I should be able to generate three data sets from random Variables A, B and C where A and B are independent and A and C are not independent.
  2. I should be able to run some test where I compare data from A and B (and A and C) and the test gives me a measure of independence.

Could I get pointers of a resource (paper, textbook chapter, code, blog post, etc.) where there is sufficient information for me to experimentally understand the concept of independence.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ What statistical language do you use? $\endgroup$
    – gung - Reinstate Monica
    Sep 15, 2013 at 21:41
  • $\begingroup$ @gung -- corrected. $\endgroup$
    – user1172468
    Sep 15, 2013 at 21:42
  • 1
    $\begingroup$ @gung, I'm kinda re-learning statistics -- I program using Java, matlab, lisp but would be open to learning R -- however I'd like to write this in a general purpose programming language such as Java $\endgroup$
    – user1172468
    Sep 15, 2013 at 21:42
  • 1
    $\begingroup$ Unless you restrict the form of dependence you're interested in, defining a single measure of dependence that has much power to detect the various forms is a pretty difficult task. For example, correlation is pretty good at finding linear dependence in the mean, but not much good at discovering the kind of dependence where the $(X,Y)$ values all lie near a circle. $\endgroup$
    – Glen_b
    Sep 15, 2013 at 23:48

2 Answers 2

Reset to default
4
$\begingroup$

I don't know Java, so someone else will have to help you with the code. In addition, I would think any decent regression textbook would cover the idea of independent random variables from a basic applied perspective, if you wanted to know about it from a mathematical perspective, any introduction to mathematical statistics should cover that fairly early on.

In a sense, the issue is hard to nail down: If two variables are not independent, there are potentially infinite ways they could be dependent. For the sake of simplicity, let's specify the type of non-independence as being linearly correlated. I'm sure Java has functions that will do this for you, but you can get a sense of how to generate linearly correlated data from my answer here: How to generate correlated random numbers (given means, variances and degree of correlation)? Likewise, you could test for (this type of) non-independence with a simple product-moment correlation test. Of course, if you were interested in some more complicated, non-linear form of correlation, then we would need to specify that in order to determine how to generate and / or test it.

Improve this answer
$\endgroup$
2
$\begingroup$

I do not recommend using correlations as a measure of independence. Basically Pearson product-moment correlation coefficient is a measure of linear correlation, which is often not the case. On the other hand Spearman's rank or Kendall tau rank correlations works also for non-linear correlated variables. However, my understanding about those procedures is that if those procedures might fail to "capture" the correlation, that does not mean at all that automatically those variables are independent.

More appropriate for this kind of task are what are called independence tests, and among them is the well-known Pearson's chi-squared test. See wiki page. I have no experience with using independence test, but the last book I studied states that is more appropriate to use G-Test. More details here on wiki page.

I currently build my own library for statistical and ml stuff and I did not yet implemented those tests, only some correlations. But I take a look on those pages and does not seem to be complex or hard to implement in any language.

I expect, however, to have some troubles generating random samples for distributions other than discrete uniform or normal Gaussian, since as far as I know, Java offers support only for those distributions.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ You're right that Pearson's $r$ will only work well for linear correlations, but the chi-squared & G-tests won't work well for continuous variables. Without specifying the form of the dependence, as @Glen_b notes above, there won't be a general way to do this. $\endgroup$
    – gung - Reinstate Monica
    Sep 16, 2013 at 1:00
  • 1
    $\begingroup$ @gung I agree, I was thinking yesterday after I commit the message that those tests won't work well for continuous variables. However I do not know if deleting a response after publish is a way to go, so I will simply correct that $\endgroup$
    – rapaio
    Sep 16, 2013 at 7:24
  • $\begingroup$ It's always OK to edit, or even delete, your posts, @Aurelian Tutuianu. You make some good points here though, so CV will be best off if you didn't delete this, but it's your choice. You could just add a caveat that this is a good strategy with categorical variables. (The truth is, I just assumed the OP meant continuous variables; nothing in the question states that.) $\endgroup$
    – gung - Reinstate Monica
    Sep 16, 2013 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.