Given the following:

  • X|θ ~ N(θ , 1/k)
  • model M0 stating that θ = θ0
  • model M1 stating that θ ~ N(u , 1/z)

How to show that X|M1 follows a normal distribution with mean u and variance 1/k + 1/z?

I think I get the intuition behind it, X is centered around θ, and θ is centered around u, so X should have expectation u as well. And the variance sums up because under M1 there's the variance for θ, and once θ is fixed there's variance around θ for X.

However, I'm struggling to formalize this intuition.

  • The law of total expectation says E[E[X|θ]] = E[X] = u, but then how do I get from E[X] = u to E[X|M1] = u?

  • Trying to calculate the marginal likelihood using the explicit PDFs was rather messy which I would prefer to avoid: f(X|M1) = ∫ f(X|θ) f(θ|M1) dθ

Thank you!


1 Answer 1


You have to start from first principles and work with definitions in order to produce a mathematically legit outcome. Intuition does not suffice.

The marginal density of the sample $X$ under model $\mathfrak M_1$ is given by the integral \begin{align} m_1(x) &= \int_\mathbb R f(x|\theta)\cdot \pi(\theta)\,\text d\theta\\ &= \int_{-\infty}^{+\infty} \dfrac{e^{-(x-\theta)^2/2}}{\sqrt{2\pi}}\cdot\dfrac{e^{-z(u-\theta)^2/2}}{\sqrt{2\pi/z}}\,\text d\theta\\ &= \int_{-\infty}^{+\infty} \dfrac{e^{-x^2/2-zu^2/2}}{\sqrt{2\pi}}\cdot\dfrac{e^{-(z+1)\theta^2/2+(x+zu)\theta}}{\sqrt{2\pi/z}}\,\text d\theta\tag{1}\\ &= \dfrac{\sqrt{z}}{\sqrt{2\pi}}e^{-x^2/2-zu^2/2}\int_{-\infty}^{+\infty} \dfrac{e^{-(\theta-(x+zu)/(z+1))^2/2+(x+zu)^2/2(z+1)}}{\sqrt{2\pi}}\,\text d\theta\\ &= \dfrac{\sqrt{z}}{\sqrt{(z+1)2\pi}}e^{-x^2/2-zu^2/2+(x+zu)^2/2(z+1)}\\&= \dfrac{\sqrt{z}}{\sqrt{(z+1)2\pi}}e^{-z(x-u)^2/2(z+1)} \end{align} Equivalently, it follows from the general result that the marginal distributions attached to a joint Normal distribution are Normal. Since $(X,\theta)$ is jointly normal (as shown by the integrand in (1)), $X$ is marginally Normal with mean $$\mathbb E[X]=\mathbb E[\mathbb E[X|\theta]]=\mathbb E[\theta]=u$$ and variance $$\mathrm{var}(X)=\mathrm{var}(\mathbb E[X|\theta])+\mathbb E[\mathrm{var}(X|\theta)]=\mathrm{var}(\theta)+\mathbb E[1]=z^{-1}+1=\dfrac{z+1}{z}$$

  • $\begingroup$ But $E[\text{var}(X|\theta)]=E[\frac{1}{k}]=\frac{1}{k}$ $\endgroup$
    – ADAM
    Feb 24 at 15:02
  • $\begingroup$ @ADAM: I hope you can make the correction for $k\ne 1$ then. $\endgroup$
    – Xi'an
    Feb 24 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.