Suppose there is a one-dimensional normal distribution $\mathcal{N}(\mu, \sigma)$ for which we want to infer the joint distribution of the parameters using Gibbs sampling. Let $D$ be the data, consisting of $n$ datapoints $d_1, ..., d_n$. Assume a broad Gamma prior to the precision parameter $\beta=\frac{1}{\sigma^2}$.

So, now I need to find an expression for $P(\sigma | D, \mu)$. Note that $P(\sigma | D, \mu) \sim P(D| \mu, \sigma) P(\sigma)$.

I tried the following:

\begin{align} P(\sigma | D, \mu) &\propto P(D| \mu, \sigma) P(\sigma) \\ &\propto \Gamma(\sigma | \alpha, \beta) \prod\limits_{i=1}^n \mathcal{N}(d_i| \mu, \sigma) \end{align}

In the next step, I throw away the $(\sqrt{2 \pi})^n$ since that is a constant.

\begin{align} \qquad\qquad\qquad\qquad\qquad\qquad &\propto_\sigma \Gamma(\sigma | \alpha, \beta) \sigma^{-n} \prod\limits_{i=1}^n \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2}{2 \sigma^2}) \\ &\propto \Gamma(\sigma | \alpha, \beta) \sigma^{-n} \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2}{2 \sigma^2}) \\ &\propto \beta^{\alpha} \sigma^{\alpha-1} \exp(-\beta \sigma) \sigma^{-n} \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2}{2 \sigma^2}) \\ &\propto \sigma^{\alpha-1-n} \exp(-\frac{\sum\limits_{i=1}^n (d_i - \mu)^2 - 2 \beta \sigma^3}{2 \sigma^2}) \end{align}

And now? Is it supposed to be another Gamma distribution? What steps am I missing? I don't see it yet.

  • $\begingroup$ Your prior on $\sigma$ or $\sigma^{-1}$ cannot involve $\sigma$ as a parameter. If you assume a $\text{G}(a,b)$ prior on $\sigma^{-2}$, the parameters $a$ and $b$ must be fixed. $\endgroup$
    – Xi'an
    Commented Oct 18, 2016 at 18:39
  • $\begingroup$ Thank you, I tried to change it, hopefully the partial answer is correct now. $\endgroup$ Commented Oct 18, 2016 at 18:47
  • 1
    $\begingroup$ Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. $\endgroup$ Commented Oct 18, 2016 at 18:56
  • 1
    $\begingroup$ The Gamma prior should be on $\sigma^{-2}$ to enjoy conjugacy. $\endgroup$
    – Xi'an
    Commented Oct 18, 2016 at 19:39

1 Answer 1


Is it supposed to be another Gamma distribution?

Yes, and that is not surprising. It is related to the concept of 'conjugate priors'. When using conjugate priors, the posterior and prior distributions of a parameter belong to the same family.

I am not sure about your equations, but if you are just looking for the answer, you can find it in this Wikipedia page: https://en.wikipedia.org/wiki/Conjugate_prior .

In the section 'Table of conjugate distributions', in 'Continuous distributions' table, look at 'Normal with known mean'. That is exactly what you need in this Gibbs sampling problem, since it gives you a solution to $P(\sigma | D, \mu)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.