3
$\begingroup$

Wikipedia says

[the inverse-chi-squared distribution] arises in Bayesian inference, where it can be used as the prior and posterior distribution for an unknown variance of the normal distribution.

Why is this distribution the one to use?

EDIT: I know about the computational convenience of conjugate priors. I don't know why inverse chi-squared is a natural one to pick for the unknown variance of the normal distribution.

EDIT 2: Let me give you an example of the type of answer I'm interested in, but with something I understand.

We often assume means are normally distributed because the central limit theorem tells us that adding together many independent, identically distributed random variables with finite non-zero variance converges to a normal distribution, and a mean is just a sum of variables (divided by a constant).

That gives me some intuition about why people assume means are normally distributed.

I have no similar intuition about why the inverse chi-squared distribution would be a natural choice to model the unknown variance of a normal distribution.

EDIT 3: For context, I saw Gelman making this assumption in this paper.

$\endgroup$
4
  • 1
    $\begingroup$ I notice you included the "conjugate-prior" tag. Does that mean you already know the inverse-chi-square is the conjugate prior in this case? $\endgroup$ – Gordon Smyth Jun 25 '20 at 8:24
  • $\begingroup$ I know about conjugate priors. I don't know why inverse chi squared should be related to the variance of a normal distribution. $\endgroup$ – dfrankow Jun 25 '20 at 14:51
  • $\begingroup$ You haven't actually answered my question. You are also inserting the word "should" where Wikipedia said "can". $\endgroup$ – Gordon Smyth Jun 25 '20 at 22:32
  • $\begingroup$ I do not know that inverse-chi-square is the conjugate prior in this case. I know someone chose it, as I just added in "EDIT 3" above. I do not know why, which is why I asked this question. $\endgroup$ – dfrankow Jun 26 '20 at 18:47
1
$\begingroup$

It is not necessarily the "one to use" in the sense that you would need to use it, of course. In fact, if your prior beliefs regarding the unknown variance differ from what is encapsulated by such a distribution, you should not use it.

However, the fact that both the prior and the posterior are from the same family tells us that this prior is a so-called conjugate prior, which may have certain advantages. These are for instance discussed in more detail here.

Conjugate priors also tend to be computationally convenient in the sense that if the posterior, generally the object of interest, follows a well-known distribution, there is a good chance we can conveniently compute posterior moments etc. and need not resort to more computer-intensive methods such as MCMC.

$\endgroup$
3
  • $\begingroup$ Thanks for your answer. I know about conjugate priors. I don't know how an inverse chi-squared distribution encodes beliefs about unknown variance in an intuitive way. Why inverse chi-squared? $\endgroup$ – dfrankow Jun 25 '20 at 14:49
  • $\begingroup$ Well, I was precisely trying to say that it need not encode anybody's beliefs, because your or my beliefs may look very different. I do not think there is anything (at least I am not aware of anything - if there is, I'd love to hear about that, though!) inherently intuitive in the shape of whatever turns out to be the conjugate prior for a given parameter. But it can serve as a prior, e.g., because it satisfies some minimal requirements for a prior for a variance like not having positive density for negative values. $\endgroup$ – Christoph Hanck Jun 25 '20 at 15:25
  • $\begingroup$ Thanks, "satisfies some minimal requirements" is useful info. So, this is one reasonable possibility. $\endgroup$ – dfrankow Jun 26 '20 at 18:49
1
$\begingroup$

It is the "conjugate prior"

If you have a look at the normal distribution, you will see that it has a density function that is proportionate (in the variance parameter) to the following form:

$$\text{N}(x|\mu,\sigma^2) \overset{\sigma^2}{\propto} \frac{1}{\sigma} \cdot \exp \bigg( -\frac{1}{2 \sigma^2} (x-\mu)^2 \bigg).$$

Similarly, if you have a look at the form of the inverse chi-squared distribution you will see that it has a density function with the proportionate form (called the "kernel" of the density):

$$\text{InvChiSq}(\sigma^2| \nu) \overset{\sigma^2}{\propto} \frac{1}{\sigma^{\nu+2}} \cdot \exp \bigg( -\frac{1}{2 \sigma^2} \bigg).$$

The similarity in these two forms means that the inverse chi-squared distribution is the "conjugate prior" for the variance parameter in the normal distribution ---i.e., use of this prior gives a posterior distribution with the same form.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.