Suppose a model,

$$x_{i} \sim N(\theta_{i}, \phi), \text{ for } i=1,\ldots,n$$

Furthermore, suppose the variance parameter, $\phi$, is some known constant.

The multidimensional Jeffreys prior is defined as,

$$\pi(\vec{\theta}) \propto \sqrt{\text{det}I(\vec{\theta})}$$

where $\text{det}I(\vec{\theta)}$ is the determinant of the multidimensional Fisher information.

Can a multidimensional Jeffreys prior be used to formulate a prior distribution for the posterior distribution, $p(\vec{\theta}|x_{1},\ldots,x_{n})$?

  • $\begingroup$ Are the $x_i$ independent from each others? $\endgroup$ – peuhp Jun 11 '14 at 12:07
  • $\begingroup$ @peuhp Yes, I meant to say that the $x_{i}$ are assumed to be i.i.d. $\endgroup$ – user9171 Jun 11 '14 at 12:31
  • $\begingroup$ I am not sure to understand. How can they be i.i.d. if they are drawn from different distributions? $\endgroup$ – peuhp Jun 11 '14 at 12:39
  • $\begingroup$ Sorry, you're absolutely right to be confused. That was a typo (bad habit, I'm afraid) - they're not identically distributed, just independent. $\endgroup$ – user9171 Jun 11 '14 at 12:43

Can a multidimensional Jeffreys prior be used to formulate a prior distribution for the posterior distribution, p(θ⃗ |x1,…,xn)?

Yes, but it's usually not recommended to use Jeffreys prior for anything other than a single parameter model. The go-to approach for multidimensional cases is the Reference prior. Reference priors maximize the K-L divergence between the prior and the posterior averaged over the sufficient statistic distribution, so that we learn as much as possible from the data. A strong introduction and justification for Reference priors can be found here. A more approachable introduction is available in this lecture accompanied by this lecture. If it's comforting at all, for single parameter models the Reference prior works out to be the Jeffreys prior in each case.


If the $x_i$ are independent from each others, I would say that the Fisher information is diagonal and you will obtain a product of the univariate Jeffreys prior which in this case are the uniform ones. So your "multivariate prior" consists of : $$ p_{\vec{\Theta}}(\vec{\theta}) \propto 1 $$

  • $\begingroup$ That's what I've ended up with, but is this a valid approach to use for the model described? In other words, is there anything theoretically invalid about using the multivariate Jeffreys prior under the conditions described above? $\endgroup$ – user9171 Jun 11 '14 at 12:53
  • $\begingroup$ @user9171 I am not an expert but I would feel confortable with doing that. To my opinion, the main question is more why building a kind of "full" model for independent variables? but I can't answer this question without understanding the context. Hope it helps. $\endgroup$ – peuhp Jun 11 '14 at 12:56
  • $\begingroup$ I agree. I'm just curious if it's valid theoretically. Thanks a million! $\endgroup$ – user9171 Jun 11 '14 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.