Why is a $p(\sigma^2)\sim\text{IG(0.001, 0.001)}$ prior on variance considered weak?

Question

Background

One of the most commonly used weak prior on variance is the inverse-gamma with parameters $\alpha =0.001, \beta=0.001$ (Gelman 2006).

However, this distribution has a 90%CI of approximately $[3\times10^{19},\infty]$.

library(pscl)
sapply(c(0.05, 0.95), function(x) qigamma(x, 0.001, 0.001))

[1] 3.362941e+19          Inf

From this, I interpret that the $IG(0.001, 0.001)$ gives a low probability that variance will be very high, and the very low probability that variance will be less than 1 $P(\sigma<1|\alpha=0.001, \beta=0.001)=0.006$.

pigamma(1, 0.001, 0.001)
[1] 0.006312353

Question

Am I missing something or is this actually an informative prior?

update to clarify, the reason that I was considering this 'informative' is because it claims very strongly that the variance is enormous and well beyond the scale of almost any variance ever measured.

follow-up would a meta-analysis of a large number of variance estimates provide a more reasonable prior?

---

Reference

Gelman 2006. Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1(3):515–533

Answer 1

Using the inverse gamma distribution, we get:

$$p(\sigma^2|\alpha,\beta) \propto (\sigma^2)^{-\alpha-1} \exp(-\frac{\beta}{\sigma^2})$$

You can see easily that if $\beta \rightarrow 0$ and $\alpha \rightarrow 0$ then the inverse gamma will approach the Jeffreys prior. This distribution is called "uninformative" because it is a proper approximation to the Jeffreys prior

$$p(\sigma^2) \propto \frac{1}{\sigma^2}$$

Which is uninformative for scale parameters see page 18 here for example, because this prior is the only one which remains invariant under a change of scale (note that the approximation is not invariant). This has a indefinite integral of $\log(\sigma^2)$ which shows that it is improper if the range of $\sigma^2$ includes either $0$ or $\infty$. But these cases are only problems in the maths - not in the real world. Never actually observe infinite value for variance, and if the observed variance is zero, you have perfect data!. For you can set a lower limit equal to $L>0$ and upper limit equal $U<\infty$, and your distribution is proper.

While it may seem strange that this is "uninformative" in that it prefers small variance to large ones, but this is only on one scale. You can show that $\log(\sigma^2)$ has an improper uniform distribution. So this prior does not favor any one scale over any other

Although not directly related to your question, I would suggest a "better" non-informative distribution by choosing the upper and lower limits $L$ and $U$ in the Jeffreys prior rather than $\alpha$ and $\beta$. Usually the limits can be set fairly easily with a bit of thought to what $\sigma^2$ actually means in the real world. If it was the error in some kind of physical quantity - $L$ cannot be smaller than the size of an atom, or the smallest size you can observe in your experiment. Further $U$ could not be bigger than the earth (or the sun if you wanted to be really conservative). This way you keep your invariance properties, and its an easier prior to sample from: take $q_{(b)} \sim \mathrm{Uniform}(\log(L),\log(U))$, and then the simulated value as $\sigma^{2}_{(b)}=\exp(q_{(b)})$.

Answer 2

It's pretty close to flat. Its median is 1.9 E298, almost the largest number one can represent in double precision floating arithmetic. As you point out, the probability it assigns to any interval that isn't really huge is really small. It's hard to get less informative than that!