3
$\begingroup$

In the wikipedia of "Student's t-distribution", "Bayesian inference" part, it is claimed that the prior distribution of the variance is taken to be $p(\sigma^{2} \mid I) \propto 1/\sigma^{2}$ but I don't quite see why. This should come as a result of entropy maximization of the prior distribution but it is not obvious to me how this can be resulted. Could you kindly help me on that?

$\endgroup$

2 Answers 2

5
$\begingroup$

The choice of a prior distribution is based on prior belief, prior information, or some constructive principle, like minimum information, maximum entropy, frequentist matching, leading to "default" or "reference" (rather than "noninformative") priors. However, there is no unique and no better/best choice for a prior as the Bayesian perspective is conditional on this prior.

In the case of a variance, $\pi(\sigma^2)=\sigma^{-2}$ is a common choice, as for instance in Jeffreys' approach or as a scale invariant (right Haar measure) prior. Again this is not the unique choice for a prior and it is not better or worse than others (unless an extra-Bayesian criterion is used to compare priors).

Even for maximum entropy priors, there is no unicity or optimality as the choice of a maximum entropy prior depends on two items of calibration:

  1. the reference measure that scales the entropy
  2. the moment constraints on the prior that lead to the functions appearing in the exponential of the density
$\endgroup$
1
  • 1
    $\begingroup$ Thank you Xi'an, this answer is exactly what I needed. I still need to digest why Jeffreys' approach or Haar measure will be justified though. I suspect, Jefferys' prior can also be written by Kullback-Leibler information, because it was explained Fisher information is taylor expantion of KL. (I think KL is used more often in modern statistics and I prefer that) $\endgroup$
    – HQMA
    Dec 2, 2019 at 8:51
0
$\begingroup$

Normal-inverse-Gamma distribution is the full conjugate prior to the normal distribution with unknown mean and variance. You can check the corresponding Wikipedia article, as well as the article on conjugate priors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.