1
$\begingroup$

I'm self-studying Bayesian Data Analysis by Gelman et al. and I'm struggling to understand the solution to exercise 2.7a. The question:

For the binomial likelihood, $y\sim Bin(n, \theta)$, show that $p(\theta) \propto \theta^{-1}(1-\theta)^{-1}$ is the uniform prior distribution for the natural parameter of the exponential family.

What I understand:

Per the question -- the natural parameter = $\phi = log(\frac{\theta}{1-\theta})$. So, $p_\phi(\phi)$ needs to be uniformly distributed. I need to show that setting $p_\theta(\theta) \propto \theta^{-1}(1-\theta)^{-1}$ results in a uniform distribution on $p_\phi$.

The solution per Gelman's website:

$q(\theta)=|\frac{d}{d\theta}log(\frac{\theta}{1-\theta})|p(\frac{e^{\phi}}{1+e^{\phi}}) \propto \theta^{-1}(1-\theta)^{-1}$

and $\frac{e^{\phi}}{1+e^{\phi}} = \theta$.

Earlier in the text we're given that $p_v(v)=|J|p_u(f^{-1}(v))$ is how one transforms a continuous R.V.

My reading of the above solution, transforming it to fit the formula above, we have $v = \phi, u = \theta$, because $f^{-1}$ here is $\frac{e^\phi}{1 + e^\phi}$, so the $p_u$ on the right takes values of $\theta$. The derivative is equal to the proposed prior, so we get the term on the right of the solution if $p(\theta) = 1$. But that doesn't answer the question, as I've understood it. Also, $q(\theta)$ on the left, takes $\theta$ as an argument, so we have two parts of this taking $\theta$ when it should be one on the right or the left. So the solution makes no sense to me. Is the answer as given on Gelman's website correct? Where am I misunderstanding what the answer is supposed to show?

$\endgroup$

1 Answer 1

0
$\begingroup$

You're almost there, but I would derive this in the other direction. You know the prior you want to specify on $\phi$, which is uniform (i.e. proportional to a constant) and want to derive the induced distribution on $\theta$ from this. So in the notation from BDA3 you have $p_v(v) = |J|p_u(f^{-1}(v)$, where $v = \theta$, $u = \phi$ and $J$ is the Jacobian of the transformation $u = f^{-1}(v)$, i.e. $\phi = f^{-1}(\theta)$. This Jacobian is (as you've already shown) equal to $\theta^{-1}(1 - \theta)^{-1}$.

Putting this all together, we find that $$ p_\theta(\theta) = |J| p_\phi(f^{-1}(\theta) = \theta^{-1}(1 - \theta)^{-1} p_\phi(f^{-1}(\theta) \propto \theta^{-1}(1 - \theta)^{-1} $$

Here the final step in the derivation follows from the fact that $p_\phi(\phi) \propto 1$.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.