1
$\begingroup$

My course notes have a problem,

Assume $\{x_1,\dots,x_n\}$ is a random sample from a Gaussian distribution with unknown mean $\mu$ and precision $\lambda$. Define the statistics $\bar{x}=\sum_{i=1}^nx_i/n$ and $s^2=n^{-1}\sum_{i=1}^n(x_i-\bar{x})^2$. Let the prior be define as $$\pi(\mu,\lambda)=\pi(\mu,\lambda)\pi(\lambda)$$ where $$\pi(\mu|\lambda)=\text{N}\left(\mu\left|m,\frac{1}{p\lambda}\right.\right) \text{ and } \pi(\lambda)=\text{Ga}(\lambda|a,b).$$ Show that the joint posterior distribution of $\mu,\lambda$ is $$\pi(\mu,\lambda|\textbf{x})=\pi_1(\mu|\lambda,\textbf{x})\pi_2(\lambda|\textbf{x}),$$ with $$\pi_1(\mu,\lambda|\textbf{x})=\text{N}\left(\mu\left|m^\ast,\frac{1}{p^\ast\lambda}\right.\right) \text{ and } \pi_2(\lambda|\textbf{x})=\text{Ga}(\lambda|a^\ast,b^\ast),$$ where \begin{align}& p^\ast=n+p,&&m^\ast=\frac{n\bar{x}+mp}{p^\ast},\\&a^\ast=a+\frac{n}{2}\text{ and }&&b^\ast=b+\frac{n}{2}\left(s^2+\frac{p}{n+p}(\bar{x}-m)^2\right).\end{align}

The course notes give a derivation using completing the square. What I can't figure out is why the $\frac{p}{n+p}(\bar{x}-m)^2$ in the $b^\ast$ isn't arbitrary. I think the completing the square involves using proportionality to discard expressions involving $n,m,\bar{x}$, so why should this particular $b^\ast$ be one that can be 'shown' to be correct?

$\endgroup$
2
  • $\begingroup$ Typo:$$m^\star=(n\bar x+pm)/p^\star$$I do not understand the question. The mathematical derivation of the posterior from the product of the prior by the likelihood is the only way to determine whether or not the expression is correct. $\endgroup$
    – Xi'an
    Jan 18 at 7:54
  • $\begingroup$ @Xi'an I've fixed the typo; thanks. My question is not about the correctness of the expression but about its uniqueness. The derivation is done up to proportionality, and I don't understand why this $b$ is chosen, or why it should be unique. $\endgroup$
    – mjc
    Jan 18 at 8:07

1 Answer 1

1
$\begingroup$

Since \begin{align}\pi(\mu|\mathbb x,\lambda)&\overbrace{\propto}^{\text{in }\mu} \exp\left[\frac{-1}{2}\left\{p\lambda(\mu-m)^2+n\lambda(\bar x-\mu)^2 \right\}\right]\\ &\propto\exp\left[\frac{-\lambda}{2}\left\{(p+n)\mu^2-2(pm+n\bar x)\mu\right\}\right]\\ &\propto\exp\left[\frac{-(n+p)\lambda}{2}(\mu-m^\star)^2\right]\\ f(\mathbf x|\lambda) &= \int f(\mathbf x|\mu,\lambda)\,\pi(\mu|\lambda)\text d\mu\\ &\underbrace{\propto}_{\text{in }\lambda}\lambda^{n/2}\exp\left[\frac{-\lambda}{2} \left\{\frac{np}{n+p}(\bar x-m^\star)^2+ns^2\right\}\right]\\ \pi(\lambda|\mathbf x) &\overbrace{\propto} \pi(\lambda)f(\mathbf x|\lambda) \\ &\propto \lambda^{a+n/2}\exp\left\{-n\lambda\left(2b+\frac{p}{n+p}(\bar x-m^\star)^2+ns^2\right)\right\}\\ \end{align} there is no ambiguity on the coefficient of $\lambda$ in the exponential.

$\endgroup$
1
  • $\begingroup$ I think it's the proportionality in the two different variables that was tripping me up. Thanks. $\endgroup$
    – mjc
    Jan 18 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.