I often hear it said that the Jeffreys prior is well-motivated because it is invariant under reparametrization. The proof of this is quite straight-forward (I know the proof on e.g., wiki). I'm a bit confused about what the proof really means, though, because the kind of invariance proven is a bit strange to me. It is indeed proven that if $$ p(x) \propto \sqrt{I(x)} $$ then $$ p(y) \propto \sqrt{I(y)} $$ where $I$ is the Fisher information and $y$ was found through a bijective transformation of $x$. Note well that $I(x)$ is an abuse of notation, as it contains derivatives wrt the variable $x$.

I don't see this as particularly compelling, since I make a similar argument that any choice of prior is parametrization invariant. E.g., by writing an arbitrary prior as $$ p(\theta) = \frac{dF(\theta)}{d\theta} $$ where $F$ is the cumulative distribution function, we then find $$ p(\phi) = \frac{dF(\phi)}{d\phi} $$ To put it another way, I can specify a prior by specifying a cdf rather than a pdf, and the cdf transforms trivially under reparameterizations. This kind of invariance is of basically no interest to me.

So, why do people make a fuss about the Jeffreys prior being invariant under reparameterization? I think I would rather say that the kind of invariance that the Jeffreys prior has is necessary for any objective formal rule for selecting a prior, but not in itself a motivation for using a Jeffreys prior. And I think it would be better to say that the Jeffreys rule for making a prior was parameterisation invariant, than say the Jeffreys prior was parameterisation invariant. Is that fair?

  • $\begingroup$ The question is how do you select the distribution $F$? $\endgroup$ – Xi'an May 6 at 7:34
  • $\begingroup$ @Xi'an For sure $F$ is totally arbitrary. I'm not suggesting this as a new rule for making priors. That was just an illustration that any prior can be written in this 'invariant' way, such that I don't see it as an advantage of the Jeffreys prior $\endgroup$ – innisfree May 6 at 7:37
  • 1
    $\begingroup$ I may have asked the same (or at least a similar) question a few years ago - if so, please check the excellent answers: stats.stackexchange.com/questions/139001/… $\endgroup$ – Christoph Hanck May 6 at 19:22
  • $\begingroup$ Thank you, certainly related but I don’t think it’s a duplicate $\endgroup$ – innisfree May 6 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.