4
$\begingroup$

In the wikipedia of "Student's t-distribution", "Bayesian inference" part, it is claimed that the prior distribution of the variance is taken to be $p(\sigma^{2} \mid I) \propto 1/\sigma^{2}$ but I don't quite see why. This should come as a result of entropy maximization of the prior distribution but it is not obvious to me how this can be resulted. Could you kindly help me on that?

$\endgroup$

2 Answers 2

7
$\begingroup$

The choice of a prior distribution is based on prior belief, prior information, or some constructive principle, like minimum information, maximum entropy, frequentist matching, leading to "default" or "reference" (rather than "noninformative") priors. However, there is no unique and no better/best choice for a prior as the Bayesian perspective is conditional on this prior.

In the case of a variance, $\pi(\sigma^2)=\sigma^{-2}$ is a common choice, as for instance in Jeffreys' approach or as a scale invariant (right Haar measure) prior. Again this is not the unique choice for a prior and it is not better or worse than others (unless an extra-Bayesian criterion is used to compare priors).

Even for maximum entropy priors, there is no unicity or optimality as the choice of a maximum entropy prior depends on two items of calibration:

  1. the reference measure that scales the entropy
  2. the moment constraints on the prior that lead to the functions appearing in the exponential of the density
$\endgroup$
1
  • 2
    $\begingroup$ Thank you Xi'an, this answer is exactly what I needed. I still need to digest why Jeffreys' approach or Haar measure will be justified though. I suspect, Jefferys' prior can also be written by Kullback-Leibler information, because it was explained Fisher information is taylor expantion of KL. (I think KL is used more often in modern statistics and I prefer that) $\endgroup$
    – HQMA
    Commented Dec 2, 2019 at 8:51
0
$\begingroup$

The right haar prior, if it exists, as it does in this case, gives predictive probability matching (see the paper by Severini et al, 2002, Biometrika). If you are interested in prediction, that's a pretty good rationale for using that prior, imho. It means that in repeated tests on your computer, or counted over all the times you use this prior in your career as a statistician, or counted across all the times anyone uses this prior in the universe today, events that are predicted to have a probability of 10% will occur 10% of the time.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.