Find the joint posterior of $(\mu, \sigma^2)$ given Normal data.

I've found the joint prior of $\mu$ and $\sigma^2$ (where $\displaystyle\sigma^2\sim\chi^{-2}(v_o,v_os_o^2)$ and $\mu\mid\sigma^2\sim N(\mu_o,\frac{\sigma^2}{n_o})$) and likelihood function:

$$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ and $$L(\mu,\sigma^2\mid x_1,\ldots,x_n)\propto (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$ respectively.

So the joint posterior of $\mu$ and $\sigma^2$ given the data is proportional to the product of both

$$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

We also know that $$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ is an inverse Gamma normal distribution with parameters $(v_o,s_o^2,\mu_o,n_o).$

I am expected to identify the joint posterior of $\mu$ and $\sigma^2$ given the data as an inverse Gamma-Normal distribution with similar parameters to $(v_o,s_o^2,\mu_o,n_o)$.

My question is which are the new parameters and what 'tricks' will I need to apply so that it has the form of inverse Gamma normal distribution?


Checking the answer given,

  • How Xi'an passed from $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ to $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$ ?

Notice that only this $n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$ changed to $n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$ though the term $v_o$ is whether a typo or what is representing? Because it's a new term in the second expression that does not exist in the first expression.

  • How can this be represented using R ? How can I plot the prior and posterior distributions, for a certain given parameters?
  • 3
    $\begingroup$ There's no trick unless you think "collect terms and simplify" and "equate coefficients of corresponding terms" are tricks. $\endgroup$
    – Glen_b
    Commented Jan 13, 2019 at 6:36
  • $\begingroup$ @Xi'an I'll edit later about that part. $\endgroup$
    – user208618
    Commented Jan 27, 2019 at 19:45
  • $\begingroup$ @Xi'an Regarding to the first part of your answer, you say that $\pi(\mu|\sigma^2)$ is proportional to an inverse Gamma with parameters $\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2)$. From here I can see the first parameter $\alpha$ but not $\beta$. More specifically, how can I rewrite $n_o(\mu-\mu_o)^2+n(\overline x-\mu)^2$ so that have the form $n_1(\mu-\mu_1) ?$ $\endgroup$
    – user208618
    Commented Jan 27, 2019 at 19:46
  • $\begingroup$ @Xi'an I think I did it, I got $(n_o+n)(\mu-\frac{n_o\mu_o+\overline xn}{n_o+n})^2+constant$. Is it correct? $\endgroup$
    – user208618
    Commented Jan 27, 2019 at 20:29
  • $\begingroup$ The first part of my answer is about $\pi(\sigma^2|\mu)$, not the reverse. And I do not exhibit a perfect square either. The explanation for $\pi(\sigma^2|\mu)$ is found in the second part and I explain there how to exhibit a perfect square for $\mu$. Which indeed leads to$$\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2$$ $\endgroup$
    – Xi'an
    Commented Jan 27, 2019 at 21:21

1 Answer 1


...the joint posterior of $\mu$ and $\sigma^2$ given the data is proportional to the product of both $$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

which simplifies into $$\displaystyle \pi(\mu,\sigma^2\mid\overline{x},s)\propto(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\,.\tag{1}$$ As a function of $\sigma^2$ only (which means conditional on everything else) this expression is proportional to an inverse Gamma conditional posterior density, with parameters $$\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2).$$ Proportional means that the above expression is missing a multiplicative constant to turn it a probability density, with integral equal to one, a constant that is called a normalisation term or a normalising constant. The full inverse Gamma density is indeed \begin{align}\Gamma(\frac{v_o+n+1}{2})^{-1}&\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{\frac{v_o+n+1}{2}}\\ &(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline{x}-\mu)^2]} \end{align} Integrating the rhs of (1) with respect to $\sigma^2$ thus produces the missing constant, $$\Gamma(\frac{v_o+n+1}{2})\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{-\frac{v_o+n+1}{2}}\tag{2}$$ and since integrating the joint posterior on $(\mu,\sigma^2)$ in $\sigma^2$ returns the marginal posterior density of $\mu$, $$\pi(\mu\mid\overline{x},s) = \int_0^\infty \pi(\mu,\sigma^2\mid\overline{x},s)\,\text{d}\sigma^2$$ this integral (2) is therefore the marginal posterior density of $\mu$, up to a multiplicative constant. With a wee bit of further work, (2) turns into a Student's $t$ density on $\mu$, with $\frac{v_o+n}{2}$ degrees of freedom, as also described in our book, Bayesian Essentials [pages 30-31].

Conversely, if integrating $\mu$ first from (1), the expression becomes proportional to $\pi(\sigma^2\mid\overline{x},s)$: since \begin{align}\pi(\mu,\sigma^2\mid\overline{x},s) &\propto (\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\overbrace{\sigma^{-1}e^{-\frac{1}{2\sigma^2}[(n_o+n)\mu^2 -2\mu(n_o\mu_o+n\overline x)]}}^\text{incomplete normal density}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\underbrace{\sigma^{-1}e^{-\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2}}_\text{conditional posterior normal density in $\mu$}\\ &\qquad\qquad\times \underbrace{e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}}_\text{missing term in perfect square}\end{align} integrating $\mu$ out returns \begin{align}\pi(\sigma^2\mid\mathbf{x}) &\propto (\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]} \end{align} since \begin{align}n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n) &=n_o\mu_o^2+n\overline{x}^2-\frac{n_o^2}{n_o+n}\mu_o^2-\frac{2n_on}{n_o+n}\mu_o\overline x-\frac{n^2}{n_o+n}\overline{x}^2\\ &= \frac{n_o\mu_o^2}{n_o+n}(n_o+n-n_o)-\frac{2n_on\mu_o\overline x}{n_o+n}+\frac{n\overline{x}^2}{n_o+n}(n_o+n-n)\\ &=\frac{nn_o}{n_o+n}(\mu_o-\overline x)^2 \end{align} which means that the marginal posterior on $\sigma^2$ is an inverse Gamma with parameters $$\frac{1}{2}(v_o+n,[v_os_o^2+(n-1)s^2+n_on(\overline x-\mu_o)^2/(n_o+n)])$$ This also follows from observing that, since $$\overline{x}\mid\mu,\sigma\sim\mathcal{N}(\mu,n^{-1}\sigma^2)\qquad \mu\mid\sigma\sim\mathcal{N}(\mu_o,n_o^{-1}\sigma^2)$$ then by marginalising in $\mu$ $$\overline{x}\mid\sigma\sim\mathcal{N}(\mu_o,[n^{-1}+n_o^{-1}]\sigma^2)$$

  • $\begingroup$ Thank you, I'm getting the point :) In your last paragraph what do you mean with 'up to a multiplicative constant.' ? $\endgroup$
    – user208618
    Commented Jan 14, 2019 at 21:40
  • $\begingroup$ How did you integrate $(1)$, did you use Mathematica or something else? $\endgroup$
    – user208618
    Commented Jan 14, 2019 at 21:46
  • $\begingroup$ And so the other term $(\sigma^2)^{-\frac{v_o+n}{2}}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s+n_o\mu_o^2+n s^2]} e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ is the marginal posterior of $\sigma^2$ right (I just want to be sure)? $\endgroup$
    – user208618
    Commented Jan 15, 2019 at 20:24
  • 1
    $\begingroup$ I was checking all again and I think there's a missing parameter in the Gamma function $\Gamma(\frac{v_o+n+1}{2})$ ? also in the Conversely part the $\sigma^{-1}$ should not be $\sigma^{-3}$ ? $\endgroup$
    – user208618
    Commented Jan 17, 2019 at 20:57
  • 1
    $\begingroup$ How did you passed from here $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ to $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$ ? I could notice the only that changed from one to the other is $n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$ to $n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$ though $v_o$ Is not in the first term. Also notice I've changed $s^2$ by $\overline x^2$ (I think it's a minor typo) $\endgroup$
    – user208618
    Commented Jan 24, 2019 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.