In order to understand the difference between the Frequentist and Bayesian inference, I was reading the presentation at: http://www.stat.ufl.edu/archived/casella/Talks/BayesRefresher.pdf . In order to explain the difference between the approaches, the author is using the following example(Page 6): If I understand correctly, there are three different cases of protocols (something related to toxicity, it seems) and there are different studies for each of the three protocols, which has examined the relation of the protocol to a specific cause, which is called something as "AAN".

For example, for one of the three protocols, different studies showing the total number of cases (first) and relation to "AAN" (second) is as following :

1) 66,11

2) 1756,129

3) 272, 48

4) 151, 18

... etc. Each of these number pairs belong to different studies.

Now, the Bayesian model for these studies are given as such:

Given $X_j$ is the AAN frequency of the $j$th study about the protocol $i$, $X_j$ is distributed as $X_j \sim Binomial(n_j,p_i)$. $n_j$ is the total number of incidences for the $j$th study and $p_i$ is the parameter of the distribution we want to infer.

What I did not understand here is, the author says that, in this model $p_i$ can vary from study to study.

In my understanding of the Bayesian approach, here, all of the studies constitute our data, $D=\left( n_1,X_1,n_2,X_2,...,n_K,X_K \right)$ where $K$ is the total number of studies. For $p_i$ we have a prior distribution of $P(p_i)$. So we try to find the posterior distribution $P(p_i|D)$. In my understanding $p_i$ cannot vary from study to study: $p_i$ is first generated from the prior distribution as $p_i \sim P(p_i)$ and then this generated value of $p_i$ is used to generate each $X_j \sim Binomial(n_j,p_i)$. Yes, $p_i$ is not fixed as in the frequentist approach, but it varies over different realizations of all $K$ studies, once a $p_i$ is generated from the prior, then all $K$ studies use the same $p_i$. So, it should not change from a single study to another. So, this slide has confused me at this particular point. Am I right with my thoughts here or did I misunderstand something?

Thanks in advance.


I think the slides are a bit ambiguous. For the Bayesian approach you can (among many other things) assume that:

  1. All protocols share the same $p$. Then there's a single prior over what that $p$ might be.

  2. Each study $i$ has its own $p_i$. Then you can either

    2.1. Assume that there's nothing in common to the three protocols, so that nothing in the results of one protocol could be informative about the others. Then you might want three independent priors, one for each $p_i$. Or, perhaps more reasonably...

    2.2 Assume that all are these three protocols are a 'sample' of the possible protocols that could have been run. In which case it's natural to think of the $p_i$ for each protocol to be a draw from a common prior over $p$. You might formalise this as reflecting an exchangeability assumption, but that doens't much matter for your question.

Assumption 1 (plus model and data) will generate a marginal posterior over $p$. Both of assumptions 2 will generate a marginal posterior over [$p_1, p_2, p_3$], but with potentially different properties.

But I suppose what the slides are going for is the idea that, from a Frequentist perspective, a natural null hypothesis to test is that $p=p_1=p_2=p_3$. Rejecting this suggests that there is variation across the $p_i$ which one might investigate further.

However one might, in equally Frequentist mode, simply ask for confidence intervals for the different $p_i$ and see if they overlap (assuming you could get the coverage right, etc.) This would give results rather closer to those generated through assumption 2.1., without formally assuming, even as a null hypothesis, that there was a single fixed $p$.

This second approach makes the Frequentist and Bayesian approaches look much more like each other. And spoils the overly sharp contrast that the slides seems to be making.

  • $\begingroup$ Well, what I understood from these slides was similar to your second model: Each protocol has their own $p_i$s. What I thought was something like that: For each protocol for their own, we have a prior distribution $P(p_i)$. Then the studies are realized, independently given $n_j$ and $p_i$, like $X_j \sim Binomial(n_j,p_i)$. I mean, it is $P(X_{1:K},|n_{1:K},p_i) = \prod_{j=1}^{K} Binomial(n_j,p_i)$. I think this corresponds to your second assumption. But the example says that $p_i$ changes for each $X_j$ which I cannot understand anyhow with this model in my mind. $\endgroup$ – Ufuk Can Bicici Aug 18 '14 at 18:53
  • $\begingroup$ Possibly these Gelman slides about an essentially identical model are helpful for understanding the 2.2 option. $\endgroup$ – conjugateprior Aug 18 '14 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.