# Predictive Posterior Distribution of Normal Distribution with Unknown Mean and Variance

Suppose that $$x_{i}|\mu,\sigma^{2} \sim \mathcal{N}(\mu,\sigma^{2})$$ for $$i = 1,...n$$. Assume that the assigned prior distributions are $$\mu$$ ~ $$\mathcal{N}$$($$\mu_{0}$$, $$\sigma^{2}_{0}$$) and $$\tau \sim Gamma(ξ_{0}, ξ_{0})$$ with $$\tau = \frac{1}{\sigma^{2}}$$ T he joint posterior distribution of $$\mu$$ and $$\tau$$, $$p(\mu,\tau|\mathbf{x})$$, where $$\mathbf{x}$$ is $$x_{1},x_{2},...,x_{n}$$, is $$p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n}{2}+ξ_{0}-1}\exp(-\tauξ_{0})\exp\left\{-\frac{1}{2}\tau\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}$$

and the full conditional distributions of $$\mu$$ and $$\tau$$ , $$p(\mu|\tau, \mathbf{x})$$ and $$p(\tau|\mu, \mathbf{x})$$ as $$p(\mu|\tau, \mathbf{x}) \sim \mathcal{N}(\frac{\tau n\bar{x}\sigma_{0}^2+\mu_{0}}{n \tau\sigma_{0}^2+1} , \frac{\sigma_{0}^{2}}{n \tau\sigma_{0}^{2}+1})$$ $$p(\tau|\mu, \mathbf{x}) \sim Gamma(\frac{n}{2}+ξ_{0},\frac{\sum_{i=1}^{n}(x_{i}-\mu)^2}{2}+ξ_{0})$$ where $$\bar{x} = \frac{\sum_{i=1}^{n}x_{i}}{n}.$$ Now I have to derive the posterior predictive distribution $$p(\tilde{x}|\mathbf{x})$$, by definition $$p(\tilde{x}|\mathbf{x}) = \int\int p(\tilde{x}|\mu,\tau) p(\mu,\tau|\mathbf{x}) d\tau d\mu$$ My problem is with the exact density of $$p(\mu,\tau|\mathbf{x})$$. It is not a Normal-Gamma distribution since it carries the term $$\exp\{-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\}$$ which does not involve $$\tau$$. Hence I cannot proceed with the integration.

If $$p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n}{2}+ξ_{0}-1}\exp(-\tauξ_{0})\exp\left\{-\frac{1}{2}\tau\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}\tag{1}$$ and (assuming $$\tilde X\sim\mathcal N(\mu,\tau^{-1})$$) $$p(\tilde{x}|\mu,\tau)\propto \tau^{1/2}{\sqrt{2\pi}}\exp\{-\frac{1}{2}\tau(\tilde x-\mu)^2\}$$ then $$p(\tilde{x}|\mu,\tau)p(\mu,\tau|\mathbf{x})\propto \tau^{\frac{n-1}{2}+\xi_{0}}\exp\left\{-\tau\xi_{0}-\frac{\tau(\tilde x-\mu)^2}{2}-\frac{\tau}{2}\sum_{i=1}^{n}(x_{i}-\mu)^2-\frac{(\mu-\mu_{0})^2}{2\sigma_{0}^{2}}\right\}$$ Integrating out $$\tau$$ gives $$p(\tilde{x}|\mu,\mathbf{x})\propto \left\{2\xi_{0}+(\tilde x-\mu)^2+\sum_{i=1}^{n}(x_{i}-\mu)^2\right\}^{-\frac{n+1}{2}-\xi_{0}}\exp\left\{-\frac{1}{2\sigma_{0}^{2}}(\mu-\mu_{0})^2\right\}$$ which has no closed form expression.
The issue stems from the choice of prior $$\mu\sim\mathcal N(\mu_0,\sigma_0^{})$$ since this prior is not conjugate: as you can see from (1), the posterior on $$\mu,\tau)$$ is not of the same form as the prior since $$\mu$$ and $$\tau$$ become dependent. A conjugate prior should involve $$\tau$$ as for instance $$\mu\sim\mathcal N(\mu_0,\omega\tau^{-1})$$ (as for instance detailed in our Bayesian essentials texbook, Chapter 2).