1
$\begingroup$

I am following the argument made by Gelman et al. in Bayesian Data Analysis (ref. in 3rd edition, p.109 onwards) for defining a non-informative prior for a hierarchical Binomial model (in the text, this is in the context of the rat tumor example):

$$ \begin{aligned} y_j & \sim \text{Bin}(N_j, \theta_j), \\ \theta_j & \sim \text{Beta}(\alpha,\beta). \end{aligned} $$

In various sources, Gelman and collaborators (references given in details below) suggest two priors for hyperparameters $\alpha,\,\beta$. The first of these is presented in BDA as

$$p(\alpha, \beta) \propto (\alpha + \beta)^{-\frac52},$$

whilst the second occurs in a number of blog posts by Gelman and collaborators/corresdpondents (here, and here), as well as the vignette for Stan and I understand is equivalent to

$$ q( \textstyle{\frac{\alpha}{\alpha+\beta}}, \alpha + \beta) \propto (\alpha + \beta)^{-\frac52}.$$

On reading the correspondence, however, I am lead to feel as though the suggestion is that these are one and the same. Whilst this is never stated explicitly, in the first blog linked to above the correspondent says (with my adjustment to TeX)

The book model reparameterized $\text{Beta}(\alpha,\beta)$ in terms of mean $\frac{\alpha}{\alpha+\beta}$ which got a uniform prior, and scale $\alpha + \beta$ with a Pareto(1.5) prior [$p(\alpha+\beta)$ proportional to $(\alpha + \beta)^{-\frac52}$].

suggesting that this prior is the same as the book model, i.e. the first one above from BDA.

My questions then are:

  1. Is there something that I am missing, are these two models the same? [See calculations below expanding on my thinking as to why not].
  2. Assuming not, the first model has a readily interpretable derivation [recapped below]; but what, if any, is the intuition for introducing the second model?

Additional notes / context.

As mentioned above, the first model has a fairly logical derivation in which they aim for a prior that is uniform on the mean $\mu$ and standard deviation $\sigma$ of the $\text{Beta}(\alpha,\beta)$ distribution. That is they introduce

$$ \begin{aligned} m & = m_{\alpha,\beta} = \frac{\alpha}{\alpha + \beta} = \mu \\ s & = s_{\alpha,\beta} = (\alpha + \beta)^{-\frac12} \approx \sigma \end{aligned} $$

and consider a uniform prior on these parameters, $\pi(m,s) \propto 1$. Using the change of variables formula one then derives the expression for $p(\alpha,\beta)$

$$ \begin{aligned} p(\alpha,\beta) & \propto \pi(m,s)\, J_{\alpha,\beta} \\ & \propto J_{\alpha,\beta} \\ & = \left| \begin{array}{cc} \partial_\alpha m_{\alpha,\beta} & \partial_\beta m_{\alpha,\beta}\\ \partial_\alpha s_{\alpha,\beta} & \partial_\beta s_{\alpha,\beta} \\ \end{array}\right| \\ & \propto (\alpha + \beta)^{-\frac52}, \end{aligned} $$ (where the term $J_{\alpha,\beta}$ denotes the Jacobian).

To justify my belief expressed in question 1) above, we can similarly perform the equivalent change of variables calculation taking as our starting point the reparameterisation proposed in the blog posts and vignette (notation taken from this later source). That is considering

$$\kappa = \alpha + \beta, \qquad \lambda = \frac{\alpha}{\alpha+\beta}$$

with the prior

$$q(\lambda, \kappa) \propto \kappa^{-\frac52}.$$

The change of variables formula now gives

$$ \begin{aligned} \tilde p(\alpha,\beta) & \propto q(\lambda,\kappa)\, J_{\alpha,\beta} \\ & \propto (\alpha + \beta)^{-\frac52} J_{\alpha,\beta} \\ & = (\alpha + \beta)^{-\frac52} \left| \begin{array}{cc} \partial_\alpha \lambda_{\alpha,\beta} & \partial_\beta \lambda_{\alpha,\beta}\\ \partial_\alpha \kappa_{\alpha,\beta} & \partial_\beta \kappa_{\alpha,\beta} \\ \end{array}\right| \\ & \propto (\alpha + \beta)^{-\frac32} \end{aligned} $$ That is, the two derived priors are clearly not equivalent when put back onto the original parameters

$$p(\alpha, \beta) \neq \tilde p(\alpha,\beta).$$


Further Notes

I am aware that Gelman has subsequently expressed a change in preference to setting the prior for this problem (discussed in the blog posts linked above). Whilst this is interesting in itself, my question is really about the mathematics of the transformations, and not the question of appropriateness of the prior.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.