# Joint posterior distribution of $(\mu,\sigma^2)$ in the Normal model

Find the joint posterior of $$(\mu, \sigma^2)$$ given Normal data.

I've found the joint prior of $$\mu$$ and $$\sigma^2$$ (where $$\displaystyle\sigma^2\sim\chi^{-2}(v_o,v_os_o^2)$$ and $$\mu\mid\sigma^2\sim N(\mu_o,\frac{\sigma^2}{n_o})$$) and likelihood function:

$$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ and $$L(\mu,\sigma^2\mid x_1,\ldots,x_n)\propto (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$ respectively.

So the joint posterior of $$\mu$$ and $$\sigma^2$$ given the data is proportional to the product of both

$$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

We also know that $$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ is an inverse Gamma normal distribution with parameters $$(v_o,s_o^2,\mu_o,n_o).$$

I am expected to identify the joint posterior of $$\mu$$ and $$\sigma^2$$ given the data as an inverse Gamma-Normal distribution with similar parameters to $$(v_o,s_o^2,\mu_o,n_o)$$.

My question is which are the new parameters and what 'tricks' will I need to apply so that it has the form of inverse Gamma normal distribution?

# Update

• How Xi'an passed from $$(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$$ to $$(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$$ ?

Notice that only this $$n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$$ changed to $$n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$$ though the term $$v_o$$ is whether a typo or what is representing? Because it's a new term in the second expression that does not exist in the first expression.

• How can this be represented using R ? How can I plot the prior and posterior distributions, for a certain given parameters?
• There's no trick unless you think "collect terms and simplify" and "equate coefficients of corresponding terms" are tricks. – Glen_b Jan 13 at 6:36
• @Xi'an I'll edit later about that part. – Isa Jan 27 at 19:45
• @Xi'an Regarding to the first part of your answer, you say that $\pi(\mu|\sigma^2)$ is proportional to an inverse Gamma with parameters $\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2)$. From here I can see the first parameter $\alpha$ but not $\beta$. More specifically, how can I rewrite $n_o(\mu-\mu_o)^2+n(\overline x-\mu)^2$ so that have the form $n_1(\mu-\mu_1) ?$ – Isa Jan 27 at 19:46
• @Xi'an I think I did it, I got $(n_o+n)(\mu-\frac{n_o\mu_o+\overline xn}{n_o+n})^2+constant$. Is it correct? – Isa Jan 27 at 20:29
• The first part of my answer is about $\pi(\sigma^2|\mu)$, not the reverse. And I do not exhibit a perfect square either. The explanation for $\pi(\sigma^2|\mu)$ is found in the second part and I explain there how to exhibit a perfect square for $\mu$. Which indeed leads to$$\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2$$ – Xi'an Jan 27 at 21:21

...the joint posterior of $$\mu$$ and $$\sigma^2$$ given the data is proportional to the product of both $$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$
which simplifies into $$\displaystyle \pi(\mu,\sigma^2\mid\overline{x},s)\propto(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\,.\tag{1}$$ As a function of $$\sigma^2$$ only (which means conditional on everything else) this expression is proportional to an inverse Gamma conditional posterior density, with parameters $$\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2).$$ Proportional means that the above expression is missing a multiplicative constant to turn it a probability density, with integral equal to one, a constant that is called a normalisation term or a normalising constant. The full inverse Gamma density is indeed \begin{align}\Gamma(\frac{v_o+n+1}{2})^{-1}&\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{\frac{v_o+n+1}{2}}\\ &(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline{x}-\mu)^2]} \end{align} Integrating the rhs of (1) with respect to $$\sigma^2$$ thus produces the missing constant, $$\Gamma(\frac{v_o+n+1}{2})\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{-\frac{v_o+n+1}{2}}\tag{2}$$ and since integrating the joint posterior on $$(\mu,\sigma^2)$$ in $$\sigma^2$$ returns the marginal posterior density of $$\mu$$, $$\pi(\mu\mid\overline{x},s) = \int_0^\infty \pi(\mu,\sigma^2\mid\overline{x},s)\,\text{d}\sigma^2$$ this integral (2) is therefore the marginal posterior density of $$\mu$$, up to a multiplicative constant. With a wee bit of further work, (2) turns into a Student's $$t$$ density on $$\mu$$, with $$\frac{v_o+n}{2}$$ degrees of freedom, as also described in our book, Bayesian Essentials [pages 30-31].
Conversely, if integrating $$\mu$$ first from (1), the expression becomes proportional to $$\pi(\sigma^2\mid\overline{x},s)$$: since \begin{align}\pi(\mu,\sigma^2\mid\overline{x},s) &\propto (\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\overbrace{\sigma^{-1}e^{-\frac{1}{2\sigma^2}[(n_o+n)\mu^2 -2\mu(n_o\mu_o+n\overline x)]}}^\text{incomplete normal density}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\underbrace{\sigma^{-1}e^{-\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2}}_\text{conditional posterior normal density in \mu}\\ &\qquad\qquad\times \underbrace{e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}}_\text{missing term in perfect square}\end{align} integrating $$\mu$$ out returns \begin{align}\pi(\sigma^2\mid\mathbf{x}) &\propto (\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]} \end{align} since \begin{align}n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n) &=n_o\mu_o^2+n\overline{x}^2-\frac{n_o^2}{n_o+n}\mu_o^2-\frac{2n_on}{n_o+n}\mu_o\overline x-\frac{n^2}{n_o+n}\overline{x}^2\\ &= \frac{n_o\mu_o^2}{n_o+n}(n_o+n-n_o)-\frac{2n_on\mu_o\overline x}{n_o+n}+\frac{n\overline{x}^2}{n_o+n}(n_o+n-n)\\ &=\frac{nn_o}{n_o+n}(\mu_o-\overline x)^2 \end{align} which means that the marginal posterior on $$\sigma^2$$ is an inverse Gamma with parameters $$\frac{1}{2}(v_o+n,[v_os_o^2+(n-1)s^2+n_on(\overline x-\mu_o)^2/(n_o+n)])$$ This also follows from observing that, since $$\overline{x}\mid\mu,\sigma\sim\mathcal{N}(\mu,n^{-1}\sigma^2)\qquad \mu\mid\sigma\sim\mathcal{N}(\mu_o,n_o^{-1}\sigma^2)$$ then by marginalising in $$\mu$$ $$\overline{x}\mid\sigma\sim\mathcal{N}(\mu_o,[n^{-1}+n_o^{-1}]\sigma^2)$$
• How did you integrate $(1)$, did you use Mathematica or something else? – Isa Jan 14 at 21:46
• And so the other term $(\sigma^2)^{-\frac{v_o+n}{2}}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s+n_o\mu_o^2+n s^2]} e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ is the marginal posterior of $\sigma^2$ right (I just want to be sure)? – Isa Jan 15 at 20:24
• I was checking all again and I think there's a missing parameter in the Gamma function $\Gamma(\frac{v_o+n+1}{2})$ ? also in the Conversely part the $\sigma^{-1}$ should not be $\sigma^{-3}$ ? – Isa Jan 17 at 20:57
• How did you passed from here $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ to $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$ ? I could notice the only that changed from one to the other is $n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$ to $n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$ though $v_o$ Is not in the first term. Also notice I've changed $s^2$ by $\overline x^2$ (I think it's a minor typo) – Isa Jan 24 at 18:25