7
$\begingroup$

Suppose that $x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$ for $i = 1,...n$. Assume that the assigned prior distributions are $\mu$ ~ $\mathcal{N}$($\mu_{0}$, $\sigma^{2}_{0}$) and $\tau \sim Gamma(ξ_{0}, ξ_{0})$ with $\tau = \frac{1}{\sigma^{2}}$ T he joint posterior distribution of $\mu$ and $\tau$, $p(\mu,\tau|\mathbf{x})$, where $\mathbf{x}$ is $x_{1},x_{2},...,x_{n}$, is $$p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n}{2}+ξ_{0}-1}\exp(-\tauξ_{0})\exp\left\{-\frac{1}{2}\tau\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}$$

and the full conditional distributions of $\mu$ and $\tau$ , $p(\mu|\tau, \mathbf{x})$ and $p(\tau|\mu, \mathbf{x})$ as $$p(\mu|\tau, \mathbf{x}) \sim \mathcal{N}(\frac{\tau n\bar{x}\sigma_{0}^2+\mu_{0}}{n \tau\sigma_{0}^2+1} , \frac{\sigma_{0}^{2}}{n \tau\sigma_{0}^{2}+1})$$ $$p(\tau|\mu, \mathbf{x}) \sim Gamma(\frac{n}{2}+ξ_{0},\frac{\sum_{i=1}^{n}(x_{i}-\mu)^2}{2}+ξ_{0})$$ where $\bar{x} = \frac{\sum_{i=1}^{n}x_{i}}{n}.$ Now I have to derive the posterior predictive distribution $p(\tilde{x}|\mathbf{x})$, by definition $$p(\tilde{x}|\mathbf{x}) = \int\int p(\tilde{x}|\mu,\tau) p(\mu,\tau|\mathbf{x}) d\tau d\mu$$ My problem is with the exact density of $p(\mu,\tau|\mathbf{x})$. It is not a Normal-Gamma distribution since it carries the term $\exp\{-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\}$ which does not involve $\tau$. Hence I cannot proceed with the integration.

$\endgroup$

1 Answer 1

1
$\begingroup$

If $$p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n}{2}+ξ_{0}-1}\exp(-\tauξ_{0})\exp\left\{-\frac{1}{2}\tau\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}\tag{1}$$ and (assuming $\tilde X\sim\mathcal N(\mu,\tau^{-1})$) $$p(\tilde{x}|\mu,\tau)\propto \tau^{1/2}{\sqrt{2\pi}}\exp\{-\frac{1}{2}\tau(\tilde x-\mu)^2\}$$ then $$p(\tilde{x}|\mu,\tau)p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n-1}{2}+\xi_{0}}\exp\left\{-\tau\xi_{0}-\frac{\tau(\tilde x-\mu)^2}{2}-\frac{\tau}{2}\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{(\mu-\mu_{0})^2}{2\sigma_{0}^{2}}\right\}$$ Integrating out $\tau$ gives $$p(\tilde{x}|\mu,\mathbf{x})\propto \left\{2\xi_{0}+(\tilde x-\mu)^2+\sum_{i=1}^{n}(x_{i}-\mu)^2\right\}^{-\frac{n+1}{2}-\xi_{0}}\exp\left\{-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}$$ which has no closed form expression.

The issue stems from the choice of prior $$\mu\sim\mathcal N(\mu_0,\sigma_0^{})$$ since this prior is not conjugate: as you can see from (1), the posterior on $\mu,\tau)$ is not of the same form as the prior since $\mu$ and $\tau$ become dependent. A conjugate prior should involve $\tau$ as for instance $$\mu\sim\mathcal N(\mu_0,\omega\tau^{-1})$$ (as for instance detailed in our Bayesian essentials texbook, Chapter 2).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.