How to derive marginal likelihood from prior and likelihood

Given the following:

• X|θ ~ N(θ , 1/k)
• model M0 stating that θ = θ0
• model M1 stating that θ ~ N(u , 1/z)

How to show that X|M1 follows a normal distribution with mean u and variance 1/k + 1/z?

I think I get the intuition behind it, X is centered around θ, and θ is centered around u, so X should have expectation u as well. And the variance sums up because under M1 there's the variance for θ, and once θ is fixed there's variance around θ for X.

However, I'm struggling to formalize this intuition.

• The law of total expectation says E[E[X|θ]] = E[X] = u, but then how do I get from E[X] = u to E[X|M1] = u?

• Trying to calculate the marginal likelihood using the explicit PDFs was rather messy which I would prefer to avoid: f(X|M1) = ∫ f(X|θ) f(θ|M1) dθ

Thank you!

The marginal density of the sample $$X$$ under model $$\mathfrak M_1$$ is given by the integral \begin{align} m_1(x) &= \int_\mathbb R f(x|\theta)\cdot \pi(\theta)\,\text d\theta\\ &= \int_{-\infty}^{+\infty} \dfrac{e^{-(x-\theta)^2/2}}{\sqrt{2\pi}}\cdot\dfrac{e^{-z(u-\theta)^2/2}}{\sqrt{2\pi/z}}\,\text d\theta\\ &= \int_{-\infty}^{+\infty} \dfrac{e^{-x^2/2-zu^2/2}}{\sqrt{2\pi}}\cdot\dfrac{e^{-(z+1)\theta^2/2+(x+zu)\theta}}{\sqrt{2\pi/z}}\,\text d\theta\tag{1}\\ &= \dfrac{\sqrt{z}}{\sqrt{2\pi}}e^{-x^2/2-zu^2/2}\int_{-\infty}^{+\infty} \dfrac{e^{-(\theta-(x+zu)/(z+1))^2/2+(x+zu)^2/2(z+1)}}{\sqrt{2\pi}}\,\text d\theta\\ &= \dfrac{\sqrt{z}}{\sqrt{(z+1)2\pi}}e^{-x^2/2-zu^2/2+(x+zu)^2/2(z+1)}\\&= \dfrac{\sqrt{z}}{\sqrt{(z+1)2\pi}}e^{-z(x-u)^2/2(z+1)} \end{align} Equivalently, it follows from the general result that the marginal distributions attached to a joint Normal distribution are Normal. Since $$(X,\theta)$$ is jointly normal (as shown by the integrand in (1)), $$X$$ is marginally Normal with mean $$\mathbb E[X]=\mathbb E[\mathbb E[X|\theta]]=\mathbb E[\theta]=u$$ and variance $$\mathrm{var}(X)=\mathrm{var}(\mathbb E[X|\theta])+\mathbb E[\mathrm{var}(X|\theta)]=\mathrm{var}(\theta)+\mathbb E[1]=z^{-1}+1=\dfrac{z+1}{z}$$
• But $E[\text{var}(X|\theta)]=E[\frac{1}{k}]=\frac{1}{k}$
• @ADAM: I hope you can make the correction for $k\ne 1$ then. Feb 24 at 15:08