0
$\begingroup$

I want to compute: $$p(x | X) = \int p(x | \mu , \Sigma) p(\mu | X) = \int \mathcal{N}(x | \mu , \Sigma) \mathcal{N}(\mu | \mu_N , \Sigma_N ) d\mu$$ Actually, this is the posterior predictive of the normal normal-mean conjugacy with kown variance. The following is the effort I have tried: \begin{align} p(x | X) &= \int p(x | \mu , \Sigma) p(\mu | X) d \mu \nonumber \\ &= \int \mathcal{N}(x | \mu , \Sigma) \mathcal{N}(\mu | \mu_N , \Sigma_N ) d\mu \nonumber \\ &= (2\pi)^{-d} |\Sigma|^{-\frac{1}{2}} |\Sigma_N|^{-\frac{1}{2}} \int exp \left( -\frac{1}{2} \left[ (x - \mu)^T \Sigma^{-1} (x - \mu) + (\mu - \mu_N)^T \Sigma_N^{-1} (\mu - \mu_N ) \right] \right) d\mu \nonumber \end{align} Denote $(x - \mu)^T \Sigma^{-1} (x - \mu) + (\mu - \mu_N)^T \Sigma_N^{-1} (\mu - \mu_N )$ as $A$, we have \begin{align} A &= \mu^T (\Sigma^{-1} + \Sigma_N^{-1}) \mu - 2\mu^T (\Sigma^{-1}x + \Sigma_N^{-1}\mu_N) + x\Sigma^{-1}x + \mu_N^T \Sigma_N^{-1}\mu_N \nonumber \\ &= [\mu - \underbrace{(\Sigma^{-1} + \Sigma_N^{-1})^{-1}(\Sigma^{-1}x + \Sigma_N^{-1}\mu_N)}_{Y}]^T (\Sigma^{-1} + \Sigma_N^{-1}) [\mu - \underbrace{(\Sigma^{-1} + \Sigma_N^{-1})^{-1}(\Sigma^{-1}x + \Sigma_N^{-1}\mu_N)}_{Y}] \nonumber \\ &\quad - || \underbrace{(\Sigma^{-1} + \Sigma_N^{-1})^{-1}(\Sigma^{-1}x + \Sigma_N^{-1}\mu_N)}_{Y}||^2 + x\Sigma^{-1}x + \mu_N^T \Sigma_N^{-1}\mu_N \nonumber \end{align} Therefore, \begin{align} p(x | X) &= (2\pi)^{-d} |\Sigma|^{-\frac{1}{2}} |\Sigma_N|^{-\frac{1}{2}} exp \left( -\frac{1}{2} \left[ -Y^T Y + x\Sigma^{-1}x + \mu_N^T \Sigma_N^{-1}\mu_N \right] \right) \nonumber \\ &\quad \int exp \left( -\frac{1}{2} \left[ (\mu - Y)^T (\Sigma^{-1} + \Sigma_N^{-1} ) (\mu - Y) \right] \right) d\mu \nonumber \\ &= (2\pi)^{-\frac{d}{2}} |\Sigma|^{-\frac{1}{2}} |\Sigma_N|^{-\frac{1}{2}} |(\Sigma^{-1} + \Sigma_N^{-1})^{-1}|^{\frac{1}{2}} exp \left( -\frac{1}{2} \left[ -Y^T Y + x\Sigma^{-1}x + \mu_N^T \Sigma_N^{-1}\mu_N \right] \right) \nonumber \end{align} Actually, The true posterior predictive is \begin{align} p(x | X) &= \mathcal{N}(x | \mu_N , \Sigma + \Sigma_N) \nonumber \\ &= (2\pi)^{-\frac{d}{2}}|\Sigma + \Sigma_N|^{-\frac{1}{2}} exp \left( -\frac{1}{2} \left[ (x - \mu_N )^T (\Sigma + \Sigma_N)^{-1} (x - \mu_N ) \right] \right) \nonumber \end{align} Therefore, the following equations should be true: \begin{align} |\Sigma|^{-\frac{1}{2}} |\Sigma_N|^{-\frac{1}{2}} |(\Sigma^{-1} + \Sigma_N^{-1})^{-1}|^{\frac{1}{2}} &= |\Sigma + \Sigma_N|^{-\frac{1}{2}} \nonumber \\ -Y^T Y + x\Sigma^{-1}x + \mu_N^T \Sigma_N^{-1}\mu_N &= (x - \mu_N )^T (\Sigma + \Sigma_N)^{-1} (x - \mu_N ) \nonumber \end{align} But I can't proof it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.