Bayes theorem an experiment design

Question

Let's say I design an experiment to test the hypothesis, "Agent Z causes cancer". Of $N$ people in the experiment group, $x$ get cancer. Of $N$ people in the control group, $y$ get cancer. The results are that $x > y$ and $N \gg x$.

Pretty standard so far. I'm interested in a Bayesian interpretation of the results of this experiment, so I come up with the following statement about the "probability that my hypothesis is correct ($P(H_{\rm true})$) given the evidence ($e$)"

$P(H_{\rm true}|e) = \frac{P(e|H_{\rm true})P(H_{\rm true})}{P(e)}$

I'm specifically interested in the quantity $P(e|H_{\rm true})$. What can be said about that quantity given the information available? Is there a general way to calculate or estimate it, knowing nothing more about the relationship between Agent Z and cancer? If not, what more would you need to calculate or estimate it?

Answer 1

First, any causality claims come from the details of the experimental setup, sampling, confounders, etc. about which Bayes, as statistical inference method has essentially nothing to say.

Nevertheless, here's a sketch: In the simplest setup there are $2N$ randomly chosen people, $N$ of whom are exposed to agent $Z$. Assume there is a baseline probability of getting cancer without $Z$ which is common to everyone involved, and the effect of $Z$ is, possibly, to raise this probability.

Your observations are then Binomial: $x$ cancers for $N$ opportunities in the treatment group and $y$ cancers in $N$ opportunities in the control group. You might then model the probability $p$ of getting cancer by relating the log odds of a getting cancer to an intercept parameter $\alpha$ that represents the baseline chance of getting cancer and a dummy variable with coefficient $\beta$ indicating being in the treatment group. This is a standard logistic regression model and it provides the likelihood.

A simple Bayesian approach to figuring out whether $\beta$ were truly positive i.e. whether $Z$ caused cancer, would be to put priors on $\alpha$ and $\beta$, compute a posterior distribution over the two unknown parameters given the data and model, and then marginalise out $\alpha$. If sufficiently much of the probability mass of the resulting posterior over $\beta$ were greater than zero you might conclude that Z caused cancer.

If you want to think in terms of $H$s, then I suppose $H_{true}$ would be that $\beta > 0$. You'd have to choose some decision rule for how much of the posterior probability would have to end up on this section of the possible $\beta$ values before declaring it true. Actually, you get rather more useful information than that from this analysis, e.g. about the range of probable effect sizes and their direction.

All in all, the resulting analysis looks pretty much like the frequentist one, but that's because I've made all the usual convenient sampling assumptions and glossed over all the actual medical issues that would make it a harder inference problem.

Answer 2

$P(e|H)$ represents a probability distribution used to model how the phenomenon "generates" observations. For example, if Z causes cancer stochastically, how would that work? Would it follow a normal distribution? You would need to make some assumption about the form of this distribution.