If I use a Jeffreys prior for a binomial probability parameter $\theta$ then this implies using a $\theta \sim beta(1/2,1/2)$ distribution.

If I transform to a new frame of reference $\phi = \theta^2$ then clearly $\phi$ is not also distributed as a $beta(1/2,1/2)$ distribution.

My question is in what sense is Jeffreys prior invariant to reparameterisations? I think I am misunderstanding the topic to be honest ...

  • 10
    $\begingroup$ Jeffreys' prior is invariant in the sense that starting with a Jeffreys prior for one parameterisation and running the appropriate change of variable is identical to deriving the Jeffreys prior directly for this new parameterisation. Actually, equivariant would be more appropriate a term than invariant. $\endgroup$
    – Xi'an
    Commented Apr 24, 2017 at 20:47
  • $\begingroup$ @ben18785: take a look at stats.stackexchange.com/questions/38962/… $\endgroup$
    – Zen
    Commented Apr 24, 2017 at 21:48
  • $\begingroup$ See also math.stackexchange.com/questions/210607/… (more or less the same question I think, but on a different site). $\endgroup$
    – N. Virgo
    Commented Apr 24, 2017 at 23:16
  • $\begingroup$ See also stats.stackexchange.com/questions/139001/… $\endgroup$ Commented Mar 6, 2019 at 7:49

1 Answer 1


Lets have $\phi = g(\theta)$, where $g$ is a monotone function of $\theta$ and let $h$ be the inverse of $g$, so that $\theta = h(\phi)$. We can obtain Jeffrey's prior distribution $p_{J}(\phi)$ in two ways:

  1. Start with the Binomial model (1) \begin{equation} \label{original} p(y | \theta) = \binom{n}{y} \theta^{y} (1-\theta)^{n-y} \end{equation} reparameterize the model with $\phi = g(\theta)$ to get $$ p(y | \phi) = \binom{n}{y} h(\phi)^{y} (1-h(\phi))^{n-y} $$ and obtain Jeffrey's prior distribution $p_{J}(\phi)$ for this model.
  2. Obtain Jeffrey's prior distribution $p_{J}(\theta)$ from original Binomial model 1 and apply the change of variables formula to obtain the induced prior density on $\phi$ $$ p_{J}(\phi) = p_{J}(h(\phi)) |\frac{dh}{d\phi}|. $$

To be invariant to reparameterisations means that densities $p_{J}(\phi)$ derived in both ways should be the same. Jeffrey's prior has this characteristic [Reference: A First Course in Bayesian Statistical Methods by P. Hoff.]

To answer your comment. To obtain Jeffrey's prior distribution $p_{J}(\theta)$ from the likelihood for Binomial model $$ p(y | \theta) = \binom{n}{y} \theta^{y} (1-\theta)^{n-y} $$ we must calculate Fisher information by taking logarithm of likelihood $l$ and calculate second derivative of $l$ \begin{align*} l := \log(p(y | \theta)) &\propto y \log(\theta) + (n-y) \log(1-\theta) \\ \frac{\partial l }{\partial \theta} &= \frac{y}{\theta} - \frac{n-y}{1-\theta} \\ \frac{\partial^{2} l }{\partial \theta^{2}} &= -\frac{y}{\theta^{2}} - \frac{n-y}{ (1-\theta)^{2} } \end{align*} and Fisher information is \begin{align*} I(\theta) &= -E(\frac{\partial^{2} l }{\partial \theta^{2}} | \theta) \\ &= \frac{n\theta}{\theta^{2}} + \frac{n - n \theta}{(1-\theta)^{2}} \\ &= \frac{n}{\theta ( 1- \theta)} \\ &\propto \theta^{-1} (1-\theta)^{-1}. \end{align*} Jeffrey's prior for this model is \begin{align*} p_{J}(\theta) &= \sqrt{I(\theta)} \\ &\propto \theta^{-1/2} (1-\theta)^{-1/2} \end{align*} which is $\texttt{beta}(1/2, 1/2)$.

  • 1
    $\begingroup$ Thanks for your answer. Afraid I am being a bit slow though. In what sense can we obtain a prior from a likelihood? They are two separate things, and the latter does not imply the former... $\endgroup$
    – ben18785
    Commented Apr 24, 2017 at 20:25
  • 4
    $\begingroup$ I answered above by obtaining a Jeffrey's prior $p_{J}(\theta)$ from the likelihood for Binomial model. $\endgroup$ Commented Apr 24, 2017 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.