2
$\begingroup$

Find the joint posterior of $(\mu, \sigma^2)$ given Normal data.

I've found the joint prior of $\mu$ and $\sigma^2$ (where $\displaystyle\sigma^2\sim\chi^{-2}(v_o,v_os_o^2)$ and $\mu\mid\sigma^2\sim N(\mu_o,\frac{\sigma^2}{n_o})$) and likelihood function:

$$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ and $$L(\mu,\sigma^2\mid x_1,\ldots,x_n)\propto (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$ respectively.

So the joint posterior of $\mu$ and $\sigma^2$ given the data is proportional to the product of both

$$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

We also know that $$\displaystyle\pi(\mu,\sigma^2)\propto(\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]}$$ is an inverse Gamma normal distribution with parameters $(v_o,s_o^2,\mu_o,n_o).$

I am expected to identify the joint posterior of $\mu$ and $\sigma^2$ given the data as an inverse Gamma-Normal distribution with similar parameters to $(v_o,s_o^2,\mu_o,n_o)$.

My question is which are the new parameters and what 'tricks' will I need to apply so that it has the form of inverse Gamma normal distribution?


Update

Checking the answer given,

  • How Xi'an passed from $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ to $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$ ?

Notice that only this $n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$ changed to $n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$ though the term $v_o$ is whether a typo or what is representing? Because it's a new term in the second expression that does not exist in the first expression.

  • How can this be represented using R ? How can I plot the prior and posterior distributions, for a certain given parameters?
$\endgroup$
  • 3
    $\begingroup$ There's no trick unless you think "collect terms and simplify" and "equate coefficients of corresponding terms" are tricks. $\endgroup$ – Glen_b Jan 13 at 6:36
  • $\begingroup$ @Xi'an I'll edit later about that part. $\endgroup$ – Isa Jan 27 at 19:45
  • $\begingroup$ @Xi'an Regarding to the first part of your answer, you say that $\pi(\mu|\sigma^2)$ is proportional to an inverse Gamma with parameters $\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2)$. From here I can see the first parameter $\alpha$ but not $\beta$. More specifically, how can I rewrite $n_o(\mu-\mu_o)^2+n(\overline x-\mu)^2$ so that have the form $n_1(\mu-\mu_1) ?$ $\endgroup$ – Isa Jan 27 at 19:46
  • $\begingroup$ @Xi'an I think I did it, I got $(n_o+n)(\mu-\frac{n_o\mu_o+\overline xn}{n_o+n})^2+constant$. Is it correct? $\endgroup$ – Isa Jan 27 at 20:29
  • $\begingroup$ The first part of my answer is about $\pi(\sigma^2|\mu)$, not the reverse. And I do not exhibit a perfect square either. The explanation for $\pi(\sigma^2|\mu)$ is found in the second part and I explain there how to exhibit a perfect square for $\mu$. Which indeed leads to$$\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2$$ $\endgroup$ – Xi'an Jan 27 at 21:21
4
+100
$\begingroup$

...the joint posterior of $\mu$ and $\sigma^2$ given the data is proportional to the product of both $$\displaystyle (\sigma^2)^{-\frac{v_o+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2]} (\sigma^2)^{-\frac{n}{2}}e^{-\frac{1}{2\sigma^2}[(n-1)s^2+n(\overline x-\mu)^2]}$$

which simplifies into $$\displaystyle \pi(\mu,\sigma^2\mid\overline{x},s)\propto(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\,.\tag{1}$$ As a function of $\sigma^2$ only (which means conditional on everything else) this expression is proportional to an inverse Gamma conditional posterior density, with parameters $$\frac{1}{2}(v_o+n+1,v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2).$$ Proportional means that the above expression is missing a multiplicative constant to turn it a probability density, with integral equal to one, a constant that is called a normalisation term or a normalising constant. The full inverse Gamma density is indeed \begin{align}\Gamma(\frac{v_o+n+1}{2})^{-1}&\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{\frac{v_o+n+1}{2}}\\ &(\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline{x}-\mu)^2]} \end{align} Integrating the rhs of (1) with respect to $\sigma^2$ thus produces the missing constant, $$\Gamma(\frac{v_o+n+1}{2})\times\left\{\frac{[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}{2}\right\}^{-\frac{v_o+n+1}{2}}\tag{2}$$ and since integrating the joint posterior on $(\mu,\sigma^2)$ in $\sigma^2$ returns the marginal posterior density of $\mu$, $$\pi(\mu\mid\overline{x},s) = \int_0^\infty \pi(\mu,\sigma^2\mid\overline{x},s)\,\text{d}\sigma^2$$ this integral (2) is therefore the marginal posterior density of $\mu$, up to a multiplicative constant. With a wee bit of further work, (2) turns into a Student's $t$ density on $\mu$, with $\frac{v_o+n}{2}$ degrees of freedom, as also described in our book, Bayesian Essentials [pages 30-31].

Conversely, if integrating $\mu$ first from (1), the expression becomes proportional to $\pi(\sigma^2\mid\overline{x},s)$: since \begin{align}\pi(\mu,\sigma^2\mid\overline{x},s) &\propto (\sigma^2)^{-\frac{v_o+n+1}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+n_o(\mu-\mu_o)^2+(n-1)s^2+n(\overline x-\mu)^2]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\overbrace{\sigma^{-1}e^{-\frac{1}{2\sigma^2}[(n_o+n)\mu^2 -2\mu(n_o\mu_o+n\overline x)]}}^\text{incomplete normal density}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2]}\underbrace{\sigma^{-1}e^{-\frac{n_o+n}{2\sigma^2}[\mu -(n_o\mu_o+n\overline x)/(n_o+n)]^2}}_\text{conditional posterior normal density in $\mu$}\\ &\qquad\qquad\times \underbrace{e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}}_\text{missing term in perfect square}\end{align} integrating $\mu$ out returns \begin{align}\pi(\sigma^2\mid\mathbf{x}) &\propto (\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}\\ &=(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]} \end{align} since \begin{align}n_o\mu_o^2+n\overline{x}^2-(n_o\mu_o+n\overline x)^2/(n_o+n) &=n_o\mu_o^2+n\overline{x}^2-\frac{n_o^2}{n_o+n}\mu_o^2-\frac{2n_on}{n_o+n}\mu_o\overline x-\frac{n^2}{n_o+n}\overline{x}^2\\ &= \frac{n_o\mu_o^2}{n_o+n}(n_o+n-n_o)-\frac{2n_on\mu_o\overline x}{n_o+n}+\frac{n\overline{x}^2}{n_o+n}(n_o+n-n)\\ &=\frac{nn_o}{n_o+n}(\mu_o-\overline x)^2 \end{align} which means that the marginal posterior on $\sigma^2$ is an inverse Gamma with parameters $$\frac{1}{2}(v_o+n,[v_os_o^2+(n-1)s^2+n_on(\overline x-\mu_o)^2/(n_o+n)])$$ This also follows from observing that, since $$\overline{x}\mid\mu,\sigma\sim\mathcal{N}(\mu,n^{-1}\sigma^2)\qquad \mu\mid\sigma\sim\mathcal{N}(\mu_o,n_o^{-1}\sigma^2)$$ then by marginalising in $\mu$ $$\overline{x}\mid\sigma\sim\mathcal{N}(\mu_o,[n^{-1}+n_o^{-1}]\sigma^2)$$

$\endgroup$
  • $\begingroup$ Thank you, I'm getting the point :) In your last paragraph what do you mean with 'up to a multiplicative constant.' ? $\endgroup$ – Isa Jan 14 at 21:40
  • $\begingroup$ How did you integrate $(1)$, did you use Mathematica or something else? $\endgroup$ – Isa Jan 14 at 21:46
  • $\begingroup$ And so the other term $(\sigma^2)^{-\frac{v_o+n}{2}}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s+n_o\mu_o^2+n s^2]} e^{\frac{1}{2\sigma^2}[(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ is the marginal posterior of $\sigma^2$ right (I just want to be sure)? $\endgroup$ – Isa Jan 15 at 20:24
  • 1
    $\begingroup$ I was checking all again and I think there's a missing parameter in the Gamma function $\Gamma(\frac{v_o+n+1}{2})$ ? also in the Conversely part the $\sigma^{-1}$ should not be $\sigma^{-3}$ ? $\endgroup$ – Isa Jan 17 at 20:57
  • 1
    $\begingroup$ How did you passed from here $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)]}$ to $(\sigma^2)^{-\frac{v_o+n}{2}-1}e^{-\frac{1}{2\sigma^2}[v_os_o^2+(n-1)s^2+n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2]}$ ? I could notice the only that changed from one to the other is $n_o\mu_o^2+n \overline x^2-(n_o\mu_o+n\overline x)^2/(n_o+n)$ to $n_ov_o(\overline x-\mu_o)^2/(n_o+v_o)^2$ though $v_o$ Is not in the first term. Also notice I've changed $s^2$ by $\overline x^2$ (I think it's a minor typo) $\endgroup$ – Isa Jan 24 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.