Say there is a hypothesis that A causes B (A -> B), and some likelihood that the hypothesis is correct (AB1%).

Now, an experiment is run that claims to find a correlation between A and B.

What I would like to know is whether my new probability (AB2%) is greater than, or less than, my previous probability (AB1%).

Common sense tells me that AB2% must be greater than AB1% -- that there's no way that finding a correlation would make it less likely that A -> B.

Is my common sense correct? I would like to account for the possibilities of sampling bias, huge experimental error, reporting bias, etc. -- everything except for falsified data.

For example, knowing that things like these can happen:

  • extremely biased experiment in which any correlation is much more likely to be due to biased sampling than to A -> B.
  • very noisy experimental design, where random correlations between noise is much more likely than observing any true correlation
  • totally random data, cherry-picked to find correlations

how would I incorporate my knowledge of those (an any other) factors into an estimate for AB2%?


Is this statement true: a correlation is found, therefore it's more likely that there's a real causal link between two variables than it was before the correlation was found?

I'm not a statistician -- please be gentle! :)

  • 1
    $\begingroup$ Have you considered Bayes' Theorem? $\endgroup$ – tristan Mar 13 '14 at 18:35
  • $\begingroup$ @tristan certainly! (thanks for pointing that out :) ) I'm just not experienced enough with it to know how to manipulate it to answer my specific question. $\endgroup$ – Matt Fenwick Mar 13 '14 at 19:13
  • $\begingroup$ In fairness a number of the complications you have suggested would not be easy to model... Reporting bias is a very difficult thing to identify/prove. I enjoyed reading Judea Pearl's book Causality which might help deal with the issue of bias... $\endgroup$ – tristan Mar 13 '14 at 19:20
  • $\begingroup$ @tristan There are lots of ways to detect spurious correlations, and these studies shouldn't be allowed to increase the likelihood of a true causal effect (i.e. don't use them). Also, correlation isn't necessarily a yes/no issue - if your prior is that there is a large correlation, and high-quality study finds a significant, but small, correlation, then this result may decrease the likelihood that the effect is as large as you previously believed. This is a nice paper on understanding Bayesian analyses, intuitively, which may help: ije.oxfordjournals.org/content/35/3/765.long $\endgroup$ – Ellie Mar 25 '14 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.