Why is a $p(\sigma^2)\sim\text{IG(0.001, 0.001)}$ prior on variance considered weak?

Background

One of the most commonly used weak prior on variance is the inverse-gamma with parameters $\alpha =0.001, \beta=0.001$ (Gelman 2006).

However, this distribution has a 90%CI of approximately $[3\times10^{19},\infty]$.

library(pscl)
sapply(c(0.05, 0.95), function(x) qigamma(x, 0.001, 0.001))

[1] 3.362941e+19          Inf


From this, I interpret that the $IG(0.001, 0.001)$ gives a low probability that variance will be very high, and the very low probability that variance will be less than 1 $P(\sigma<1|\alpha=0.001, \beta=0.001)=0.006$.

pigamma(1, 0.001, 0.001)
[1] 0.006312353


Question

Am I missing something or is this actually an informative prior?

update to clarify, the reason that I was considering this 'informative' is because it claims very strongly that the variance is enormous and well beyond the scale of almost any variance ever measured.

follow-up would a meta-analysis of a large number of variance estimates provide a more reasonable prior?

Reference

Gelman 2006. Prior distributions for variance parameters in hierarchical models. Bayesian Analysis 1(3):515–533

-
 A "true" noninformative prior is not a distribution. So there's no prior probability such as P(sigma<1). – Stéphane Laurent Dec 11 '12 at 12:33

Using the inverse gamma distribution, we get:

$$p(\sigma^2|\alpha,\beta) \propto (\sigma^2)^{-\alpha-1} \exp(-\frac{\beta}{\sigma^2})$$

You can see easily that if $\beta \rightarrow 0$ and $\alpha \rightarrow 0$ then the inverse gamma will approach the Jeffrey's prior. This distribution is called "uninformative" because it is a proper approximation to the Jeffrey's prior

$$p(\sigma^2) \propto \frac{1}{\sigma^2}$$

Which is uninformative for scale parameters see page 18 here for example, because this prior is the only one which remains invariant under a change of scale (note that the approximation is not invariant). This has a indefinite integral of $\log(\sigma^2)$ which shows that it is improper if the range of $\sigma^2$ includes either $0$ or $\infty$. But these cases are only problems in the maths - not in the real world. Never actually observe infinite value for variance, and if the observed variance is zero, you have perfect data!. For you can set a lower limit equal to $L>0$ and upper limit equal $U<\infty$, and your distribution is proper.

While it may seem strange that this is "uninformative" in that it prefers small variance to large ones, but this is only on one scale. You can show that $\log(\sigma^2)$ has an improper uniform distribution. So this prior does not favor any one scale over any other

Although not directly related to your question, I would suggest a "better" non-informative distribution by choosing the upper and lower limits $L$ and $U$ in the Jeffreys prior rather than $\alpha$ and $\beta$. Usually the limits can be set fairly easily with a bit of thought to what $\sigma^2$ actually means in the real world. If it was the error in some kind of physical quantity - $L$ cannot be smaller than the size of an atom, or the smallest size you can observe in your experiment. Further $U$ could not be bigger than the earth (or the sun if you wanted to be really conservative). This way you keep your invariance properties, and its an easier prior to sample from: take $q_{(b)} \sim \mathrm{Uniform}(\log(L),\log(U))$, and then the simulated value as $\sigma^{2}_{(b)}=\exp(q_{(b)})$.

-
+1 for not only answering the question, but also providing useful advice. – whuber Mar 10 '11 at 4:57
+1 - uniform for $log(\sigma)$ on a "big" range is often a good choice. For the variance components of a hierarchical model I think you can still get into near-impropriety of the posterior if the range is too large since you're approaching Jeffrey's again. But of course it's a simpler fix - just don't pick huge intervals :) – JMS Mar 10 '11 at 5:04
@JMS - in a heirarchical setting, the data do not "squash" the singularity at 0 (i.e. a level 2 variance could be zero). So the prior for small values matters. $Beta_{2}(1,1)$ is a good level 2 and higher variance prior (I think it also has been called a "half Cauchy", it is similar to $F_{1,1}$-distribution). It has "fat tails" and is "data-robust" in that, if prior and likelihood conflict, the likelihood wins. Also $Beta_{2}(0,0)$ is the jeffreys prior. – probabilityislogic Mar 10 '11 at 5:58
@probabilityislogic thanks for the explanation. If I understand, the gamma is nice theoretically because its rage is $[0,\infty]$ and because it is conjugate to the normal, but in application these features are not generally required. But what is the difference between sampling from $\sigma\sim exp(U(log(L),log(U))$ and $\sigma\sim U(L,U)$? – David Mar 10 '11 at 15:41
@probabilityislogic Not familiar with your notation, are you referring to the beta prime? If so it's an interesting choice. Not the half Cauchy though; that's just the Cauchy restricted to $(0, \infty)$. But the beta prime with $\alpha=1, \beta=1/2$ has been called the "quasi Cauchy" IIRC – JMS Mar 10 '11 at 16:36