Joint posterior distribution of $(\mu,\sigma^2)$ in the Normal model

Question

Find the joint posterior of $(\mu, \sigma^2)$ given Normal data.

I've found the joint prior of $\mu$ and $\sigma^2$ (where $\displaystyle\sigma^2\sim\chi^{-2}(v_o,v_os_o^2)$ and $\mu\mid\sigma^2\sim N(\mu_o,\frac{\sigma^2}{n_o})$) and likelihood function:

$$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ and $$L(\mu,\sigma^2\mid x_1,\ldots,x_n)\propto (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$ respectively.

So the joint posterior of $\mu$ and $\sigma^2$ given the data is proportional to the product of both

$$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

We also know that $$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ is an inverse Gamma normal distribution with parameters $(v_o,s_o^2,\mu_o,n_o).$

I am expected to identify the joint posterior of $\mu$ and $\sigma^2$ given the data as an inverse Gamma-Normal distribution with similar parameters to $(v_o,s_o^2,\mu_o,n_o)$.

My question is which are the new parameters and what 'tricks' will I need to apply so that it has the form of inverse Gamma normal distribution?

---

Update

Checking the answer given,

• How Xi'an passed from $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ to $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$ ?

Notice that only this $n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$ changed to $n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$ though the term $v_o$ is whether a typo or what is representing? Because it's a new term in the second expression that does not exist in the first expression.

• How can this be represented using R ? How can I plot the prior and posterior distributions, for a certain given parameters?

Answer 1

...the joint posterior of $\mu$ and $\sigma^2$ given the data is proportional to the product of both $$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

which simplifies into $$\displaystyle \pi(\mu,\sigma^2\mid\overline{x},s)\propto(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\,.\tag{1}$$ As a function of $\sigma^2$ only (which means conditional on everything else) this expression is proportional to an inverse Gamma conditional posterior density, with parameters $$\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2).$$ Proportional means that the above expression is missing a multiplicative constant to turn it a probability density, with integral equal to one, a constant that is called a normalisation term or a normalising constant. The full inverse Gamma density is indeed \begin{align}\Gamma(\frac{v_o+n+1}{2})^{-1}&\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{\frac{v_o+n+1}{2}}\\ &(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline{x}-\mu)^2]} \end{align} Integrating the rhs of (1) with respect to $\sigma^2$ thus produces the missing constant, $$\Gamma(\frac{v_o+n+1}{2})\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{-\frac{v_o+n+1}{2}}\tag{2}$$ and since integrating the joint posterior on $(\mu,\sigma^2)$ in $\sigma^2$ returns the marginal posterior density of $\mu$, $$\pi(\mu\mid\overline{x},s) = \int_0^\infty \pi(\mu,\sigma^2\mid\overline{x},s)\,\text{d}\sigma^2$$ this integral (2) is therefore the marginal posterior density of $\mu$, up to a multiplicative constant. With a wee bit of further work, (2) turns into a Student's $t$ density on $\mu$, with $\frac{v_o+n}{2}$ degrees of freedom, as also described in our book, Bayesian Essentials [pages 30-31].

Conversely, if integrating $\mu$ first from (1), the expression becomes proportional to $\pi(\sigma^2\mid\overline{x},s)$: since \begin{align}\pi(\mu,\sigma^2\mid\overline{x},s) &\propto (\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\overbrace{\sigma^{-1}e^{-\frac{1}{2\sigma^2}[(n_o+n)\mu^2 -2\mu(n_o\mu_o+n\overline x)]}}^\text{incomplete normal density}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\underbrace{\sigma^{-1}e^{-\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2}}_\text{conditional posterior normal density in $\mu$}\\ &\qquad\qquad\times \underbrace{e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}}_\text{missing term in perfect square}\end{align} integrating $\mu$ out returns \begin{align}\pi(\sigma^2\mid\mathbf{x}) &\propto (\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]} \end{align} since \begin{align}n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n) &=n_o\mu_o^2+n\overline{x}^2-\frac{n_o^2}{n_o+n}\mu_o^2-\frac{2n_on}{n_o+n}\mu_o\overline x-\frac{n^2}{n_o+n}\overline{x}^2\\ &= \frac{n_o\mu_o^2}{n_o+n}(n_o+n-n_o)-\frac{2n_on\mu_o\overline x}{n_o+n}+\frac{n\overline{x}^2}{n_o+n}(n_o+n-n)\\ &=\frac{nn_o}{n_o+n}(\mu_o-\overline x)^2 \end{align} which means that the marginal posterior on $\sigma^2$ is an inverse Gamma with parameters $$\frac{1}{2}(v_o+n,[v_os_o^2+(n-1)s^2+n_on(\overline x-\mu_o)^2/(n_o+n)])$$ This also follows from observing that, since $$\overline{x}\mid\mu,\sigma\sim\mathcal{N}(\mu,n^{-1}\sigma^2)\qquad \mu\mid\sigma\sim\mathcal{N}(\mu_o,n_o^{-1}\sigma^2)$$ then by marginalising in $\mu$ $$\overline{x}\mid\sigma\sim\mathcal{N}(\mu_o,[n^{-1}+n_o^{-1}]\sigma^2)$$