Posterior Predictive Distribution of a Parameter

Question

Suppose I have data, $y_1, \ldots, y_n \sim \mathcal{N}(\mu, \sigma^2)$, where $\mu$ is an unknown parameter and $\sigma^2$ is known. We put a prior on the parameter $\mu \sim \mathcal{N}(\mu_0, \tau^2_0)$. Under these assumptions one can compute the posterior

$$ \begin{aligned} p(\mu_n &| y_1, \ldots, y_n; \mu_0, \sigma^2) \sim \mathcal{N}(\alpha_n, \tau_n^2) \\ \alpha_n &= \frac{\frac{\mu_0}{\tau^2_0} + \frac{n\bar{y}}{\sigma^2}}{ \left(\frac{1}{\tau^2_0} + \frac{n}{\sigma^2}\right)},\quad \tau_n^2 = \left(\frac{1}{\tau^2_0} + \frac{n}{\sigma^2}\right)^{-1} \end{aligned} $$

and the posterior predictive of a new data sample $y_{n+1}$

$$ p(y_{n+1} | y_1, \ldots, y_n; \mu_0, \sigma^2) \sim \mathcal{N}\left(\alpha_n, \sigma^2 +\tau_n^2\right) $$

I am interested in computing the posterior predictive distribution of the parameter (propagating uncertainty from the posterior predictive of the data to the posterior predictive of the parameter)

I started by writing

$$ \begin{aligned} p(\mu_{n+1} &| y_1, \ldots, y_n, y_{n+1}; \mu_0, \sigma^2) \sim\mathcal{N}(\alpha_{n+1}, \tau_{n+1}^2) \\ \alpha_{n+1} &= \frac{\frac{\alpha_n}{\tau^2_n} + \frac{y_{n+1}}{\sigma^2}}{ \left(\frac{1}{\tau^2_n} + \frac{1}{\sigma^2}\right)},\quad \tau_{n+1}^2 = \left(\frac{1}{\tau^2_n} + \frac{1}{\sigma^2}\right)^{-1} \end{aligned} $$

$$ \begin{aligned} p(\mu_{n+1}| y_1, \ldots, y_n; \mu_0, \sigma^2) = \int p(\mu_{n+1} &| y_1, \ldots, y_n, y_{n+1}; \mu_0, \sigma^2) p(y_{n+1} | y_1, \ldots, y_n; \mu_0, \sigma^2) dy_{n+1} \\ \end{aligned} $$

However, there is a $y_{n+1}$ in the mean which interacts with $\mu_{n+1}$ and is not separable. So I cannot compute an analytical posterior.

I can draw samples $\tilde{y}^i_{n+1}$ from the posterior predictive distribution of the data and construct an approximate average $\frac{1}{S} \sum_{i=1}^S p(\mu_{n+1} | y_1, \ldots, y_n, y^i_{n+1}; \mu_0, \sigma^2)$ but I want to know if someone has seen an example of this type with an analytical posterior.

How does one propagate uncertainty from the posterior predictive of the data to the posterior predictive of the parameter. Is there any other way to approximate except sampling. I would appreciate any search terms, examples or references

Answer 1

There is no parameter $\mu_n$ or $\mu_{n+1}$ in your model, only $\mu$ which distribution gets updated as you get more data observed. The posterior predictive is defined for future observations not for a parameter. The integrated posterior of $\mu$ given $(y_1,\ldots,y_{n+1})$ is simply the posterior of $\mu$ given $(y_1,\ldots,y_{n})$ as shown by inverting the order of conditioning in $\mu$ and $y_{n+1}$: \begin{align} p(\mu &| y_1, \ldots, y_n, y_{n+1}; \mu_0, \sigma^2) p(y_{n+1} | y_1, \ldots, y_n; \mu_0, \sigma^2)\\ &= p(\mu | y_1, \ldots, y_n; \mu_0, \sigma^2) p(y_{n+1} | y_1, \ldots, y_n,\mu; \mu_0, \sigma^2)\\ &= p(\mu | y_1, \ldots, y_n; \mu_0, \sigma^2) p(y_{n+1} | \mu; \mu_0, \sigma^2) \end{align}