# The PDF of the Data Given (Marginal Likelihood) the Likelihood and the Prior of a Normal Distribution with Prior on the Mean

Given a model where $$x_i | \mu \sim \mathcal{N} ( \mu, \sigma^2 )$$ where $$\mu \sim \mathcal{N} ( \mu_0, \sigma_0^2 )$$, is there a closed form formula for the PDF of $$x_i$$? Namely, what's $$p (x_i)$$?

I know the solution by Bayes, but I wonder if there is a closed form solution. My intuition is a Normal distribution with updated mean and variance according to the prior.

One way to model this would be by a sum of 2 variables:

$${x}_{i} = {y}_{i} + {z}_{i}, \quad {y}_{i} \sim \mathcal{N} \left( 0, {\sigma}_{2}^{2} \right), \; {z}_{i} \sim \mathcal{N} \left( {\mu}_{0}, {\sigma}_{0}^{2} \right)$$

Since $${z}_{i} \perp {y}_{i}$$ then the variance of $${x}_{i}$$ is the sum of variances.
Hence $${x}_{i} \sim \mathcal{N} \left( {\mu}_{0}, {\sigma}^{2} + {\sigma}_{0}^{2} \right)$$.

Do it with moment-generating functions and iterated expectations. We have:

\begin{align*} M_X(t) & = \mathbb{E}[\exp(tX)] = \mathbb{E}[\mathbb{E}(\exp(tX)|\mu)] \\ & = \mathbb{E}[\exp(\mu t + \sigma^2 t^2 / 2 )] = (\sigma^2 t^2 / 2) \, \mathbb{E}[\exp(\mu t)] \\ & =(\sigma^2 t^2 / 2)\, \exp(\mu_0 t + \sigma_0^2 t^2 / 2 ) \\ & = \exp\{\mu_0 t + (\sigma^2+\sigma_0^2) t^2 / 2 \}, \end{align*}

which is the MGF of a $$\mathcal{N}(\mu_0, \sigma^2 +\sigma_0^2)$$.

• This is nice! I had a different way in mind. +1.
– Royi
Apr 22, 2022 at 16:10
• @Royi well it is essentially the proof that you can add Gaussian variables and get another Gaussian variable
– qwr
Apr 22, 2022 at 21:05