How would I, experimentally prove that two random variables drawn either from the same population or two different populations are independent (or for that matter of fact dependent).
$\begingroup$
$\endgroup$
2
asked Nov 5, 2012 at 13:37
-
1$\begingroup$ Some tools for checking possible relationship (note that they do not offer a "proof", just some clues): scatter plots, correlation, $\chi^2$ test. $\endgroup$– user10525Commented Nov 5, 2012 at 14:08
-
3$\begingroup$ I don't think you can prove this. @Procrastinator offers some tools to give evidence, but really this is a question that revolves around how the variables were drawn, because even independent samples will be correlated sometimes (indeed, if you randomly generate two variables many times and check correlation, e.g., they will be significantly correlated 5% of the time). $\endgroup$– Peter FlomCommented Nov 5, 2012 at 15:05
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
@Procrastinator has offered good tools to check a possible relationship.
If you really want to get mathematical, get ready because this is actually a very very deep problem. You should first estimate their dependency by some sort of correlation estimator. Then, you'll do inference on that estimate. That is, you will do a hypothesis test to see if that correlation is equal to zero or not.
I can think of two things:
1) Assume that they come from a multivariate normal distribution. Then, they are independent if and only if Pearson's correlation is equal to zero. However, know that testing correlation is a bitch. You will really need a large sample size for the test. Look at the graph here to get some ideas. Caveat: Even if sample 1 and sample 2 has a normal distribution, this doesn't mean that they come from a multivariate normal distribution. Someone else might elaborate more on this.
2) This question was bothering Gabor Szekely for a long long time. He recently come up with the solution: distance correlation. Again, you will have to estimate the distance correlation from your sample. But the good thing is that, distance correlation is actually equal to zero if your samples are actually independent. This helps you to get around the multivariate normal assumption. You should estimate the distance correlation from the sample, and then do hypothesis testing as you would have done with a multivariate normal.
-
$\begingroup$ Does "independence" in this context already imply lack of a linear relationship? If not, you could have two variables that have a 100% deterministic relationship, but in ways that won't show up in a linear correlation (e.g., a quadratic relationship that's symmetrically distributed around the mean). $\endgroup$– octernCommented Nov 5, 2012 at 17:28
-
1$\begingroup$ This is probably obvious, but I'd add that for real-world observations, you're testing for lack of a significant difference from zero at some level of confidence. Real populations have a finite size, so chance variations are not guaranteed to average out exactly to zero, even in the total absence of any relationship between the variables. $\endgroup$– octernCommented Nov 5, 2012 at 17:38
-
$\begingroup$ @octern, in the multivariate Gaussian case, independence == no relationship. Regarding distance correlation: it still detects those nasty relationships. But your point is very valid, pearson's correlation only checks for a linear relationship. There are other types of estimators like Spearman's correlation and distance correlation that perform better when yo have nonlinear relationships. $\endgroup$– SvBCommented Nov 5, 2012 at 18:15
-
$\begingroup$ Spearman's rho does indeed check for nonlinear relationships, but it should be emphasized that it checks for monotonic nonlinear relationships. $\endgroup$– Kiran K.Commented Oct 2, 2016 at 18:27