The PDF of the Data Given (Marginal Likelihood) the Likelihood and the Prior of a Normal Distribution with Prior on the Mean

Question

Given a model where $ x_i | \mu \sim \mathcal{N} ( \mu, \sigma^2 ) $ where $ \mu \sim \mathcal{N} ( \mu_0, \sigma_0^2 ) $, is there a closed form formula for the PDF of $ x_i $? Namely, what's $ p (x_i) $?

I know the solution by Bayes, but I wonder if there is a closed form solution. My intuition is a Normal distribution with updated mean and variance according to the prior.

Answer 1

One way to model this would be by a sum of 2 variables:

$$ {x}_{i} = {y}_{i} + {z}_{i}, \quad {y}_{i} \sim \mathcal{N} \left( 0, {\sigma}_{2}^{2} \right), \; {z}_{i} \sim \mathcal{N} \left( {\mu}_{0}, {\sigma}_{0}^{2} \right) $$

Since $ {z}_{i} \perp {y}_{i} $ then the variance of $ {x}_{i} $ is the sum of variances.
Hence $ {x}_{i} \sim \mathcal{N} \left( {\mu}_{0}, {\sigma}^{2} + {\sigma}_{0}^{2} \right) $.

Answer 2

Do it with moment-generating functions and iterated expectations. We have:

$$ \begin{align*} M_X(t) & = \mathbb{E}[\exp(tX)] = \mathbb{E}[\mathbb{E}(\exp(tX)|\mu)] \\ & = \mathbb{E}[\exp(\mu t + \sigma^2 t^2 / 2 )] = (\sigma^2 t^2 / 2) \, \mathbb{E}[\exp(\mu t)] \\ & =(\sigma^2 t^2 / 2)\, \exp(\mu_0 t + \sigma_0^2 t^2 / 2 ) \\ & = \exp\{\mu_0 t + (\sigma^2+\sigma_0^2) t^2 / 2 \}, \end{align*} $$

which is the MGF of a $\mathcal{N}(\mu_0, \sigma^2 +\sigma_0^2)$.