# Posterior Predictive Distribution - Normal model with unknown mean, and known std dev

I do not understand how one can prove that in a normal model, where we know the variance, but not the mean, the posterior predictive distribution is also normal...

Below is cut of a page from Gelman's Data Bayesian Analysis. I don't understand how the authors can state that the posterior predictive dist. will be a Normal one, i.e., why can they conclude that it's a joint normal posterior distribution and then conclude that the marginal is also normal? Any help would be appreciated.

## 2 Answers

if $X \sim N(\mu_X, \sigma_X^2)$ and $Y \sim N(\mu_Y, \sigma_Y^2)$ and $X$ and $Y$ are independent then $$f_Z(z) = \int_{-\infty}^\infty f_Y(z-x) f_X(x) dx$$ is a normal density with mean $\mu_X+\mu_Y$ and covariance $\sigma^2_X+\sigma_Y^2$ (that is actually the density of $Z=X+Y$). Simple proof at https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables#Proof_using_convolutions which states your result.

• Thanks for your answer peuhp. Actually, I don't think there's need for a change of variable. In the picture above, the $\theta$ is your $x$. What do you reckon? – An old man in the sea. Mar 29 '16 at 15:09
• Happy to help. You completely right. I correct this – peuhp Mar 29 '16 at 15:11

If the likelihood is normal, i.e. $$y_i|\theta \sim N(\theta,\sigma_x^2)$$ and the prior is normal, i.e. $$\theta_{prior} \sim N(\theta_0, \sigma_{\theta}^2)$$, then the posterior is also normal, $$\theta \sim N(\mu_1, \tau_1^2)$$. You can follow a proof here.

Now, the predictive distribution will also be normal:

$$P(y'|y) = \int_{\theta}P(y', \theta |y)d\theta = \int_{\theta}P(y'|y,\theta)P(\theta|y)d\theta = \int_{\theta}P(y'|\theta)P(\theta|y)d\theta$$

The first step is just treating the probability as a marginal probability over the parameter space. The 2nd step is just the conditional probability. The 3rd step is the assumption that once we know $$\theta$$, it's the only thing that influences the new observation, and any old observation is useless. What you end up with is the integral over all $$\theta$$ of the likelihood times the posterior.

You could calculate these integrals and see that you end up with a normal distribution. Or you can use a little trick, using convolutions (that peuhp hints of):

$$\int_{\theta}P(y' = y_0|\theta)P(\theta|y)d\theta = \int_{\theta}P(y' = y_0 - \theta+\theta|\theta)P(\theta|y)d\theta = \int_{\theta}P(y' = y_1 + \theta|\theta)P(\theta|y)d\theta = \int_{\theta}P(y'-\theta = y_1|\theta)P(\theta|y)d\theta = \int_{\theta}P(y'-\theta|\theta)P(\theta|y)d\theta$$

Here $$y_0$$ is the value that the random variable is equal to. I used variable change and called $$y_1 = y_0-\theta$$.

Note that what we end up with is a convolution of two functions which belong to two distributions: $$\theta$$ which we already know; and $$y-\theta |\theta \sim N(0, \sigma_x^2)$$. I.e. what we get is the sum of these two normal distributions, which is equal to:

$$y'|y \sim N(\mu_1, \sigma_x^2 + \tau_1^2)$$