Merits of reparameterizing the Gamma and inverse Gamma

Question

Wikipedia states that the PDFs for the Gamma distribution is:

$$ f(x|\alpha,\beta) = \frac{\beta^\alpha}{\Gamma(\alpha)}x^{\alpha-1}\exp(-\beta x) $$

However, in Rasmussen 2000, the pdf for the Gamma distribution is specified as: $$ p(r) \tilde{} \mathcal{G}(1,\sigma_y^{-2}) \propto r^{-\frac{1}{2}}\exp\Big(-\frac{r}{2\sigma_y^{-2}}\Big) $$ Which means that the pdf used is $$ f(x|a,b)\propto x^{\frac{a}{2}-1}\exp\Big(-\frac{x}{2b}\Big) $$

---

We have the same case for the inverse Gamma distribution, which in Wikipedia is:

$$ f(x|\alpha,\beta) = \frac{\beta^\alpha}{\Gamma(\alpha)}x^{-\alpha-1}\exp\Big(-\frac{\beta}{x}\Big) $$

While Rasmussen 2000 specifies: $$ p(x^{-1})~\mathcal{G}(1,1) \implies p(x)\propto x^{-\frac{3}{2}}\exp\Big(-\frac{1}{2x}\Big) $$

Which means that the pdf used is either

$$f(x|a,b)\propto x^{-\frac{a}{2} - 1}\exp\Big(-\frac{1}{2bx}\Big)$$

or

$$f(x|a,b)\propto x^{-\frac{a}{2} - 1}\exp\Big(-\frac{b}{2x}\Big)$$

---

Because Rasmussen's paper only specifies the PDF up to a normalization constant, the reparameterization of the PDF is acceptable.

However, what is the rationale for specifying the PDF in this manner? Does this make it easier to differentiate with respect to $x$ or the parameters $a,b$, or do properties such as log-concavity arise or something (especially since we're possibly taking $1/b$ instead of $\beta$ in the exponentials)?

Answer 1

Rasmussen selects a vague prior by the provided selection of parameters, see the following quote from the paper you mentioned:

The shape parameter of the Gamma prior is set to unity, corresponding to a very broad (vague) distribution

Similar thing happens to Inverse Gamma prior.

The reason behind selection of a vague prior is to make the prior less informative and reduce effects of the prior on the posterior. It is called Objective Bayesian approach to the prior selection.

P.S. There is a theory behind selection of the most noniformative prior e.g. reference and Jeffreys priors try to reduce ammount of information in the prior.

Answer 2

My guess is that Rasmussen in that paper sees the Gamma distribution as the particular one-dimensional case of the Wishart distribution (also here). From this point of view, if the probability density of $x$ has a Gamma distribution then $$ \mathrm{p}(x \mid a, b)\ \mathrm{d}x \propto x^{\frac{a}{2}-1}\ \exp\Bigl(-\frac{x}{2 b}\Bigr)\ \mathrm{d}x \ , $$ where the parameters $a$ and $b$ are the degrees of freedom and the (1D) scale matrix of the Wishart. Rasmussen calls $a$ "shape"; I think it would have been better to call it "degrees of freedom".

But this is not the end of the story.

Rasmussen calls the second parameter "mean". The mean $m$ of the 1D Wishart is $a b$, and if the second parameter is the mean then we have $$ \mathrm{p}(x \mid a, m)\ \mathrm{d}x \propto x^{\frac{a}{2}-1}\ \exp\Bigl(-\frac{a x}{2 m}\Bigr)\ \mathrm{d}x $$ instead. Unfortunately Rasmussen only writes the Gammas explicitly when $a=1$, in which case $m=b$, so it isn't clear whether he's using the first or second parametrizations above. But checking the second of equations (5) for the case $k=1$, obtained by multiplying (2) and the second of (3), confirms that the last parametrization above is the one Rasmussen is using.

So the conversion between Rasmussen's parameters $\alpha_{\text{R}}$, $\beta_{\text{R}}$ in his "$\mathcal{G}(\alpha_{\text{R}}, \beta_{\text{R}})$" and the commoner shape $\alpha$ and rate $\beta$ of the Gamma $\mathrm{G}(x \mid \alpha, \beta)\ \mathrm{d}x \propto x^{\alpha-1}\ \exp(-\beta x)\ \mathrm{d}x$, is as follows: $$ \left\{\begin{aligned} \alpha &= \frac{\alpha_{\text{R}}}{2}\\ \beta &= \frac{\alpha_{\text{R}}}{2 \beta_{\text{R}}} \end{aligned}\right. \qquad \left\{\begin{aligned} \alpha_{\text{R}} &= 2 \alpha\\ \beta_{\text{R}} &= \frac{\alpha}{\beta} \end{aligned}\right. $$

Analogously for the inverse Gamma seen as the particular 1D case of the inverse Wishart.