# Parameterization of inverse gamma prior in Bayesian methods

For a prior of $$\sigma^2 \sim IG(0.01, 0.01)$$, often recommended as an uninformative prior for the variance parameter in MCMC approaches and other Bayesian methods, which parameterization does this recommendation correspond to?

Is it shape/scale or shape/rate? That is, in R, would it be invgamma::dinvgamma(sigma, shape = 0.01, scale = 0.01) or invgamma::dinvgamma(sigma, shape = 0.01, rate = 0.01)?

TLDR: The recommendation for an Inverse Gamma prior on variance parameters uses the shape/scale parameterization. But take care with the invgamma package which seems to have mixed up rate and scale.

I can see why the parameterization is confusing if you use the invgamma package: In its documentation the Inverse Gamma density is written as

f(x) = rate^shape/Gamma(shape) x^(-1-shape) e^(-rate/x)

In the Wikipedia article on the Inverse Gamma distribution, the pdf is of course the same: $$f(x) = \beta^\alpha/\Gamma(\alpha) x^{-\alpha-1}\exp(-\beta/x)$$ but the $$\beta$$ parameter is described as the scale.

Wikipedia gets it right: $$\beta$$ is the "scale", not the "rate" of the Inverse Gamma. This is the standard parameterization.

It may be helpful to reason about this by simply plotting the density. Note that I use the invgamma package which seems to have mixed up scale and rate.

The pdf on the right doesn't look like the density of a "noninformative" prior as it puts all its mass on large values of $$x$$. So if you use the invgamma package (better not?), you would specify the "rate"; if you use another implementation, you would specify the scale.

And for more up-to-date prior recommendations, take a look at the Stan documentation: Prior Choice Recommendations. This guide has five levels of priors, from flat to specific informative.

For example the recommendations for a weakly informative prior on a scale parameter, such as the variance, include an exponential with expected value 10, a half-normal (0,10) and a half-Cauchy(0,5).

library("invgamma")

par(mfrow = c(1, 2))
curve(
dinvgamma(x, shape = 0.01, rate = 0.01),
main = "shape = 0.01, rate = 0.01",
from = 0.0001, to = 1.5, n = 10001, font.main = 1,
xlim = c(0, 1.5), ylab = "f(x)", yaxs = "i"
)
curve(
dinvgamma(x, shape = 0.01, scale = 0.01),
main = "shape = 0.01, scale = 0.01",
from = 0.0001, to = 1.5, n = 10001, font.main = 1,
xlim = c(0, 1.5), ylab = "f(x)", yaxs = "i"
)

• this was incredibly clear and helpful, especially with the code and graphs you have provided. I appreciate it @dipetkov
– bob
Feb 9 at 16:18

The inverse gamma is a conjugate prior for $$\sigma^2$$ in typical Gaussian-data models. The underlying idea is to have a convenient proper-prior approximation to an improper $$\sigma^{-2}$$ prior, for a weakly informative/low information prior. In the limit as $$\epsilon=\beta=\alpha$$ goes to $$0$$, the $$\text{inverse gamma}(\epsilon, \epsilon)$$ distribution goes to that improper prior. This occurs when $$\beta$$ is a scale parameter, not a rate parameter.

This use goes back at least to about 1993 give or take, that was when I first encountered it - I believe in conversation - and indeed subsequently used it in an illustrative example. Gelman [1] credits Spiegelhalter et al (1994), so that's possibly its first appearance, though I would not be in the least surprised to discover it's a good deal older than that.

A warning: according to Gelman ([1], again) the $$\text{inverse gamma}(\epsilon, \epsilon)$$ for small $$\epsilon$$ (where 0.01 or 0.001 are typical choices of small) distribution is not as weakly informative a prior as was hoped for; he says that the results depend on which $$\epsilon$$ is chosen.

[1]: Gelman, A., (2006),
"Prior distributions for variance parameters in hierarchical models,"
Bayesian Analysis 1, No. 3, pp. 515–533
http://www.stat.columbia.edu/%7Egelman/research/published/taumain.pdf