# Inference using Gibbs sampling

Suppose there is a one-dimensional normal distribution $\mathcal{N}(\mu, \sigma)$ for which we want to infer the joint distribution of the parameters using Gibbs sampling. Let $D$ be the data, consisting of $n$ datapoints $d_1, ..., d_n$. Assume a broad Gamma prior to the precision parameter $\beta=\frac{1}{\sigma^2}$.

So, now I need to find an expression for $P(\sigma | D, \mu)$. Note that $P(\sigma | D, \mu) \sim P(D| \mu, \sigma) P(\sigma)$.

I tried the following:

\begin{align} P(\sigma | D, \mu) &\propto P(D| \mu, \sigma) P(\sigma) \\ &\propto \Gamma(\sigma | \alpha, \beta) \prod\limits_{i=1}^n \mathcal{N}(d_i| \mu, \sigma) \end{align}

In the next step, I throw away the $(\sqrt{2 \pi})^n$ since that is a constant.

\begin{align} \qquad\qquad\qquad\qquad\qquad\qquad &\propto_\sigma \Gamma(\sigma | \alpha, \beta) \sigma^{-n} \prod\limits_{i=1}^n \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2}{2 \sigma^2}) \\ &\propto \Gamma(\sigma | \alpha, \beta) \sigma^{-n} \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2}{2 \sigma^2}) \\ &\propto \beta^{\alpha} \sigma^{\alpha-1} \exp(-\beta \sigma) \sigma^{-n} \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2}{2 \sigma^2}) \\ &\propto \sigma^{\alpha-1-n} \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2 - 2 \beta \sigma^3}{2 \sigma^2}) \end{align}

And now? Is it supposed to be another Gamma distribution? What steps am I missing? I don't see it yet.

• Your prior on $\sigma$ or $\sigma^{-1}$ cannot involve $\sigma$ as a parameter. If you assume a $\text{G}(a,b)$ prior on $\sigma^{-2}$, the parameters $a$ and $b$ must be fixed. – Xi'an Oct 18 '16 at 18:39
• Thank you, I tried to change it, hopefully the partial answer is correct now. – www.data-blogger.com Oct 18 '16 at 18:47
• Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. – gung - Reinstate Monica Oct 18 '16 at 18:56
• The Gamma prior should be on $\sigma^{-2}$ to enjoy conjugacy. – Xi'an Oct 18 '16 at 19:39

In the section 'Table of conjugate distributions', in 'Continuous distributions' table, look at 'Normal with known mean'. That is exactly what you need in this Gibbs sampling problem, since it gives you a solution to $P(\sigma | D, \mu)$.