Why is the inverse chi-squared distribution a natural prior and posterior for an unknown variance of a normal distribution?

Question

Wikipedia says

[the inverse-chi-squared distribution] arises in Bayesian inference, where it can be used as the prior and posterior distribution for an unknown variance of the normal distribution.

Why is this distribution the one to use?

EDIT: I know about the computational convenience of conjugate priors. I don't know why inverse chi-squared is a natural one to pick for the unknown variance of the normal distribution.

EDIT 2: Let me give you an example of the type of answer I'm interested in, but with something I understand.

We often assume means are normally distributed because the central limit theorem tells us that adding together many independent, identically distributed random variables with finite non-zero variance converges to a normal distribution, and a mean is just a sum of variables (divided by a constant).

That gives me some intuition about why people assume means are normally distributed.

I have no similar intuition about why the inverse chi-squared distribution would be a natural choice to model the unknown variance of a normal distribution.

EDIT 3: For context, I saw Gelman making this assumption in this paper.

Answer 1

It is not necessarily the "one to use" in the sense that you would need to use it, of course. In fact, if your prior beliefs regarding the unknown variance differ from what is encapsulated by such a distribution, you should not use it.

However, the fact that both the prior and the posterior are from the same family tells us that this prior is a so-called conjugate prior, which may have certain advantages. These are for instance discussed in more detail here.

Conjugate priors also tend to be computationally convenient in the sense that if the posterior, generally the object of interest, follows a well-known distribution, there is a good chance we can conveniently compute posterior moments etc. and need not resort to more computer-intensive methods such as MCMC.

Answer 2

It is the "conjugate prior"

If you have a look at the normal distribution, you will see that it has a density function that is proportionate (in the variance parameter) to the following form:

$$\text{N}(x|\mu,\sigma^2) \overset{\sigma^2}{\propto} \frac{1}{\sigma} \cdot \exp \bigg( -\frac{1}{2 \sigma^2} (x-\mu)^2 \bigg).$$

Similarly, if you have a look at the form of the inverse chi-squared distribution you will see that it has a density function with the proportionate form (called the "kernel" of the density):

$$\text{InvChiSq}(\sigma^2| \nu) \overset{\sigma^2}{\propto} \frac{1}{\sigma^{\nu+2}} \cdot \exp \bigg( -\frac{1}{2 \sigma^2} \bigg).$$

The similarity in these two forms means that the inverse chi-squared distribution is the "conjugate prior" for the variance parameter in the normal distribution ---i.e., use of this prior gives a posterior distribution with the same form.