5
$\begingroup$

Working through the book Bayesian Essentials with R by Jean-Michel Marin & Christian Robert I am trying to work out the posterior for the model given on page 29 when the data is from a normal with unknown mean and variance.

The priors for $\mu$ and $\sigma^2$ are (p.29-30)

\begin{align} \pi(\mu | \sigma^2) &\sim N(0,\sigma^2)\\ \pi(\sigma^2) &\sim \exp(-\sigma^{-2})(\sigma^2)^{-2} \end{align}

and the likelihood (from top of page 29) as

$$l \propto \exp(-(n(\mu - \bar{x})^2 + s^2)/2\sigma^2)/\sigma^n$$

On the top of page 31 the posterior is given as

$$\pi((\mu, \sigma^2) | D) \propto \pi(\sigma^2) \times \pi(\mu | \sigma^2) \times l$$

Plugging the values in gives

\begin{align} \pi((\mu, \sigma^2) | D) \propto ~ & \exp(-\sigma^{-2})(\sigma^2)^{-2} \times \\ &\exp(-\mu^2 / 2\sigma^2) \times \exp(-(n(\mu - \bar{x})^2 + s^2)/2\sigma^2)/\sigma^n\tag{1} \end{align}

I cannot see how to move from this to what is given below.

enter image description here

I'd really appreciate any hints. Thank you.


I can combined the exponent parts of the prior on $\sigma$ and $\mu$ to give $\exp(-\mu^2/2\sigma^2 - 1/\sigma^2)$ which is equal to $\exp(-(\mu^2 + 2)/2\sigma^2)$. I can see how the $1/\sigma^n$ from the likelihood is equal to $(\sigma^{-2})^{n/2}$ but I cannot see how $(\sigma^2)^{-2} = \sigma^{-4}$ is equal to $(\sigma^{-2})^{1/2 + 2}$ , which I think equals $\sigma^{-5}$.

$\endgroup$

1 Answer 1

5
$\begingroup$

There is a typo¹ in the exponential part if not in the power: Starting from \begin{align} \pi((\mu, \sigma^2) | D) \propto ~ & \overbrace{\exp(-\sigma^{-2})(\sigma^2)^{-2}}^{\pi(\sigma^2)} \times \overbrace{\sigma^{-1} \exp(-\mu^2 / 2\sigma^2)}^{\pi(\mu|\sigma^2)} \times \\ &\underbrace{\exp(-(n(\mu - \bar{x})^2 + s^2)/2\sigma^2)/\sigma^n}_{\ell(\mu,\sigma^2|D)} \end{align} one gets \begin{align} \pi((\mu, \sigma^2) | D) \propto ~ & (\sigma^{-2})^{\overbrace{\frac{1}{2}+2+\frac{n}{2}}^{(n+5)/2}}\times\\ &\exp\left\{-[(n+1)\mu^2-2n\mu\bar x+n\bar{x}^2+s^2+2]/2\sigma^2\right\}\\ \propto ~ & (\sigma^{-2})^{\frac{n+2}{2}+1+\frac{1}{2}}\times\\ &\exp\left\{-(n+1)(\mu-n\bar{x}/(n+1))^2\right\}\times\\ &\exp\left\{-[n\bar{x}^2/(n+1)+2+s^2]/2\sigma^2\right\}\\ = ~ & (\sigma^{-2})^{\frac{1}{2}} \exp\left\{-(n+1)(\mu-n\bar{x}/(n+1))^2\right\}\times\\ &(\sigma^{-2})^{\frac{n+2}{2}+1}\exp\left\{-[n\bar{x}^2/(n+1)+2+s^2]/2\sigma^2\right\} \end{align} Hence, compared with the derivation at the top

  1. the powers of $\sigma^2$ are indeed $-1/2$ and $-(n+2)/2-1$: $$(\sigma^{-2})^{\frac{n+5}{2}}=(\sigma^2)^\frac{-1}{2}\times (\sigma^2)^{-\frac{n+2}{2}-1}$$
  2. there is a missing $n\bar{x}^2/(n+1)$ term in the scale parameter of the posterior inverse Gamma distribution. (See Murphy's derivation for double-check.)

Apologies for the confusion!


¹It was already pointed out in earlier X validated questions:

Computing posterior density of Normal with unknown $\mu$ and $\sigma^2$

Help with Bayesian derivation of normal model with conjugate prior

$\endgroup$
4
  • $\begingroup$ Dear Xi'an. Thank you for your answer and my apologies that I did not find the previous questions.Can i check from where did the additional $\sigma^{-1}$ come from in the "Starting from". Is it from the normalising bit in the prior on $\pi(\mu|\sigma^2)$? e.g. $\pi(\mu|\sigma^2) \propto \sigma^{-1}exp(-\mu^2/2\sigma^2)$. Thank you. $\endgroup$ Commented May 27, 2021 at 15:46
  • $\begingroup$ Indeed, there is a $\sigma^{-1}$ missing from your equation (1) which is due to the prior $\pi(\mu|\sigma^2)$ . $\endgroup$
    – Xi'an
    Commented May 27, 2021 at 17:19
  • 2
    $\begingroup$ Thank you very much Xi'an. I am sure you must yearn for more interesting questions but it really is top class of you to give your time to teach. $\endgroup$ Commented May 27, 2021 at 17:24
  • 1
    $\begingroup$ Sorry I misunderstood your question for a while. The reason $\sigma^{-1}$ must be included is that $\pi(\mu,\sigma^2)$ is a joint probability density. $\endgroup$
    – Xi'an
    Commented May 27, 2021 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.