Parametrisation invariance/covariance of the Jeffreys prior

Question

I've been trying to understand what exactly is meant by parametrisation invariance of the Jeffreys prior.

Already I've read here that invariance is technically not the best term to use, and that it's more a case of covariance. My understanding of covariance is that it describes the property of transforming in a particular known way, which agrees with the `change of variables theorem' that I've often seen invoked in the context of reparametrising a Jeffreys prior.

$$p(\theta) = p(\psi) \left| \frac{d\psi}{d\theta} \right|$$

I presume this comes from setting the probability area of $p(\theta) d\theta$ equal to $p(\psi) d\psi$ (and then essentially dividing by the infinitesimal $d\theta$), though I'm kind of wary of such expressions in the absence of integrals.

My question then is how this gives parametrisation invariance/covariance. For example, the Jeffreys prior $p(\sigma) = \frac{1}{\sigma}$ over the positive reals is known to be invariant/covariant under power transformations. Choosing $\gamma = \sigma^n$ and applying this change of variables theorem however, I get

$$p(\sigma) = p(\gamma) \left| \frac{d\gamma}{d\sigma} \right| = \frac{1}{\sigma^n} n \sigma^{n-1} = \frac{n}{\sigma} \neq p(\sigma)$$

so I must be doing something wrong, unless it's simply a matter of normalisation.

Answer 1

It is simply a matter of normalization. The improper prior distribution is $$p(\sigma) \propto 1/\sigma \\\propto n/\sigma,$$ which bears out the claim of invariance under power transformations.

In general, we can we can ignore constants when characterizing closed form probability distributions. For example, to derive the posterior distribution for $\sigma^2$ when $Y_1, ..., Y_n \stackrel{iid}{\sim} N(m, \sigma^2)$ (known mean) under this Jeffrey's prior: $$ p(\sigma^2|y) \propto p(\sigma^2)\times p(Y|\sigma)\\ \propto 1/\sigma^2 \times 1/\sigma^n \times \exp\left\{\frac{-1}{\sigma^2} \; \frac{\sum_{i=1}^{n}(y_i-m)^2}{2}\right\}\\ =\left(\sigma^2\right)^{-n/2 - 1}\times\exp\left\{\frac{-1}{\sigma^2} \; \frac{\sum_{i=1}^{n}(y_i-m)^2}{2}\right\},$$ which implies that $\sigma^2|Y$ is $IG\left(\frac{n}{2}, \frac{\sum_{i=1}^{n}(y_i-m)^2}{2}\right)$.