Chapter 5 of Bayesian Data Analysis 3rd Edition uses an example of rat endometrial stromal polyps to illustrate the concept of hierarchical regression.

In particular, Gelman and coauthors go on to compute the posterior of the following model of 71 binomial observations:

$$y_i \sim \operatorname{Bin}(\theta_i; n_i)$$ $$\theta_i \sim \operatorname{Beta}(\alpha, \beta)$$ $$p(\alpha,\beta) \propto \alpha \beta (\alpha + \beta)^{-5/2}$$

Gelman writes something I find a little confusing on page 112 when describing how one may simulate from the posterior distribution:

For each $j = 1, \dots ,71$, sample $\theta_j$ from its conditional posterior distribution $\theta_j \vert \alpha, \beta, y \sim \operatorname{Beta}(\alpha+ y_j, \beta+n_j - y_j)$.


Once I obtain posterior estimates for alpha and beta, can I not interpret the thetas as coming from a beta distribution with those shape parameters?

What is the difference in interpretation between $\theta_j \vert \alpha, \beta, y \sim \operatorname{Beta}(\alpha+ y_j, \beta+n_j - y_j)$ and a beta distirnution parameterized with the posterior estimates for alpha and beta?


Intuitively, the point of a hierarchical model is that we want the $(\theta_j)$ to be similar (they share a prior distribution), but not equal; this is explained in BDA section 5.5 (pp. 119-121) for example. In particular, we want the data $y_j$ to inform on $\theta_j$. The marginal posterior distribution of $\theta_j$ will end up somewhere between the distribution we would have got with a model estimating each effect separately, and the distribution we would have got with a pooled estimate.

If you were to follow your suggestion, then you would not allow each $\theta_j$ to be specifically informed by $y_j$, and so you would essentially be using a pooled estimate.

| cite | improve this answer | |
  • $\begingroup$ So am I right to interpret the model as follows: that each $\theta_j$ come from a beta distribution with parameters alpha and beta, and that the alpha and beta are obtained from the marginal posterior (i.e. the mean of 5.8)? $\endgroup$ – Demetri Pananos Nov 26 '18 at 12:53
  • $\begingroup$ There is no closed form for the marginal posterior of $\theta_j$. Conditionally on the values of $\alpha$ and $\beta$, the $\theta_j$ come from a Beta distribution. However, $\alpha$ and $\beta$ are not known exactly: we have a distribution for them as well. To get a sample from the distribution of $\theta_j$, you need to draw samples from the marginal posterior of $(\alpha, \beta)$, and then for each of those samples draw from the conditional posterior $\theta_j\sim Beta(\alpha+y_j,\beta+n_j-y_j)$. $\endgroup$ – Robin Ryder Nov 26 '18 at 14:31
  • $\begingroup$ You would have $\theta_{72}|\alpha,\beta\sim Beta(\alpha, \beta)$. To get a realization of $\theta_{72}$, sample $\alpha$ and $\beta$ from the posterior, and then $\theta_{72}$ from this conditional distribution. $\endgroup$ – Robin Ryder Nov 26 '18 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.