0
$\begingroup$

Suppose I have data, $y_1, \ldots, y_n \sim \mathcal{N}(\mu, \sigma^2)$, where $\mu$ is an unknown parameter and $\sigma^2$ is known. We put a prior on the parameter $\mu \sim \mathcal{N}(\mu_0, \tau^2_0)$. Under these assumptions one can compute the posterior

$$ \begin{aligned} p(\mu_n &| y_1, \ldots, y_n; \mu_0, \sigma^2) \sim \mathcal{N}(\alpha_n, \tau_n^2) \\ \alpha_n &= \frac{\frac{\mu_0}{\tau^2_0} + \frac{n\bar{y}}{\sigma^2}}{ \left(\frac{1}{\tau^2_0} + \frac{n}{\sigma^2}\right)},\quad \tau_n^2 = \left(\frac{1}{\tau^2_0} + \frac{n}{\sigma^2}\right)^{-1} \end{aligned} $$

and the posterior predictive of a new data sample $y_{n+1}$

$$ p(y_{n+1} | y_1, \ldots, y_n; \mu_0, \sigma^2) \sim \mathcal{N}\left(\alpha_n, \sigma^2 +\tau_n^2\right) $$

I am interested in computing the posterior predictive distribution of the parameter (propagating uncertainty from the posterior predictive of the data to the posterior predictive of the parameter)

I started by writing

$$ \begin{aligned} p(\mu_{n+1} &| y_1, \ldots, y_n, y_{n+1}; \mu_0, \sigma^2) \sim\mathcal{N}(\alpha_{n+1}, \tau_{n+1}^2) \\ \alpha_{n+1} &= \frac{\frac{\alpha_n}{\tau^2_n} + \frac{y_{n+1}}{\sigma^2}}{ \left(\frac{1}{\tau^2_n} + \frac{1}{\sigma^2}\right)},\quad \tau_{n+1}^2 = \left(\frac{1}{\tau^2_n} + \frac{1}{\sigma^2}\right)^{-1} \end{aligned} $$

$$ \begin{aligned} p(\mu_{n+1}| y_1, \ldots, y_n; \mu_0, \sigma^2) = \int p(\mu_{n+1} &| y_1, \ldots, y_n, y_{n+1}; \mu_0, \sigma^2) p(y_{n+1} | y_1, \ldots, y_n; \mu_0, \sigma^2) dy_{n+1} \\ \end{aligned} $$

However, there is a $y_{n+1}$ in the mean which interacts with $\mu_{n+1}$ and is not separable. So I cannot compute an analytical posterior.

I can draw samples $\tilde{y}^i_{n+1}$ from the posterior predictive distribution of the data and construct an approximate average $\frac{1}{S} \sum_{i=1}^S p(\mu_{n+1} | y_1, \ldots, y_n, y^i_{n+1}; \mu_0, \sigma^2)$ but I want to know if someone has seen an example of this type with an analytical posterior.

How does one propagate uncertainty from the posterior predictive of the data to the posterior predictive of the parameter. Is there any other way to approximate except sampling. I would appreciate any search terms, examples or references

$\endgroup$
1
$\begingroup$

There is no parameter $\mu_n$ or $\mu_{n+1}$ in your model, only $\mu$ which distribution gets updated as you get more data observed. The posterior predictive is defined for future observations not for a parameter. The integrated posterior of $\mu$ given $(y_1,\ldots,y_{n+1})$ is simply the posterior of $\mu$ given $(y_1,\ldots,y_{n})$ as shown by inverting the order of conditioning in $\mu$ and $y_{n+1}$: \begin{align} p(\mu &| y_1, \ldots, y_n, y_{n+1}; \mu_0, \sigma^2) p(y_{n+1} | y_1, \ldots, y_n; \mu_0, \sigma^2)\\ &= p(\mu | y_1, \ldots, y_n; \mu_0, \sigma^2) p(y_{n+1} | y_1, \ldots, y_n,\mu; \mu_0, \sigma^2)\\ &= p(\mu | y_1, \ldots, y_n; \mu_0, \sigma^2) p(y_{n+1} | \mu; \mu_0, \sigma^2) \end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.