3
$\begingroup$

I have found two different expressions for the Jeffreys prior of a multivariate Gaussian. Eq. (3) in this article states that $$p(\mu,\Sigma) \propto \det(\Sigma)^{-(d+2)/2}$$

However in page 73 of this book it is claimed that $$p(\mu,\Sigma) \propto \det(\Sigma)^{-(d+1)/2}$$

Furthermore, in the first article it says that the proposed density

[...] is the only density which makes the Fisher information for the parameters invariant to all possible reparameterizations of the Gaussian.

Wouldn't this mean that the second prior cannot be Jeffreys, by definition? Is the second one wrong then?

$\endgroup$
5
  • $\begingroup$ There's some useful discussion here for the $d=1$ case: stats.stackexchange.com/questions/156199/… $\endgroup$ Feb 3 at 6:20
  • $\begingroup$ Where did you find $d+2$? Gelman defines the prior density as proportional to $|\Sigma|^{-(d+1) /2}$ whereas that paper defines the same as $|\mathbf V|^{-(d+1) /2}.$ $\endgroup$ Feb 3 at 6:33
  • $\begingroup$ @User1865345 The paper in my question says $$p(\mathbf{m},\mathbf{V}) \propto |2\pi \mathbf{V}|^{-1/2}|\mathbf V|^{-(d+1) /2} = (2\pi)^{-d/2}|\mathbf{V}|^{-1/2}|\mathbf{V}|^{-(d+1) /2}$$ The last expression is proportional to $|\mathbf{V}|^{-1/2}|\mathbf{V}|^{-(d+1) /2}=|\mathbf{V}|^{-(d+2) /2}$. $\endgroup$
    – Tendero
    Feb 3 at 14:21
  • $\begingroup$ @ThomasLumley I don't think the issue discussed in that question would arise in the multivariate case, as this distribution is always defined in terms of the covariance matrix (the confusion between variance and standard deviation could never happen). $\endgroup$
    – Tendero
    Feb 3 at 14:35
  • $\begingroup$ I looked at a glance, so missed the part @Tendero. $\endgroup$ Feb 3 at 17:40

1 Answer 1

4
$\begingroup$

Gelman is not wrong. One needs to understand the context.

For Inverse-Wishart distribution $\mathcal{IW}(\mathbf X;\mathbf \Psi,\nu),$ the relevant term of the density is $$ |\mathbf X|^{-\frac{(\nu+d+1)}2}\exp\left\{-\frac12\operatorname{tr}\left(\mathbf \Psi\mathbf X^{-1}\right)\right\}\tag 1\label 1$$ where $d$ is the order of $\mathbf X, ~\mathbf \Psi.$

The author of the paper takes $\mathcal{IW}(\mathbf V;k\mathbf V_0,k)$ and so $\eqref 1$ becomes as $k\to 0,$

$$ |\mathbf V|^{-\frac{(d+1)}2};$$ therefore $$p(\mathbf m, \mathbf V) \propto |\mathbf V|^{-\frac12}|\mathbf V|^{-\frac{(d+1)}2}=|\mathbf V|^{-\left(\frac{d}2+1\right)}.\tag 2$$

Gelman whereas considers $\mathcal{IW}\left(\mathbf \Sigma;\mathbf \Lambda_0, \nu_0\right) $ and then takes $k_0\to 0,~ \nu_0\to-1, ~\det(\mathbf \Lambda_0) \to 0,$ (note previously $k$ was tending to $0$ but here its counterpart $\nu_0$ is tending to $-1$) to get

$$p(\boldsymbol\mu, \mathbf \Sigma)\propto |\mathbf\Sigma|^{-\frac{(d+1)}2}.\tag 3$$

In fact, as this paper notes, both are valid objective priors, the latter being the independence-Jeffreys prior.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.