I am referring to the book "Pattern recognition and machine learning" book by Bishop. Equation 2.104 states that the precision matrix for the joint distribution is

$ \begin{equation} R = \left(\begin{array}{cc} \Lambda+A^TLA & -A^TL\\ -LA & L \end{array}\right) \end{equation} $

and the mean of the joint distribution is

$ \begin{equation} E[z] = \left(\begin{array}{cc} \mu\\ A\mu+b\end{array}\right) \end{equation} $

For conditional Gaussian distributions equation 2.75 states that

$ \mu_{a|b} = \mu_{a} - \Lambda_{aa}^{-1}\Lambda_{ab}(x_{b} - \mu_{b}) $

From the equation for R

$ \Lambda_{aa}^{-1} = (\Lambda+A^TLA)^{-1} $

$ \Lambda_{ab} = -A^TL $

$ \mu_{a} = mu $

and $ \mu_{b} = -A\mu + b $

Substituting in expression for $\mu_{a|b}$ I get

$ E[x|y] = \mu + (\Lambda+A^TLA)^{-1}(A^TL)(y - (A\mu + b)) $ But this expression is different from what is concluded in equation 2.111 which is

$ E[x|y] = (\Lambda+A^TLA)^{-1} [A^TL(y - b) + \Lambda\mu] $

How do I reconcile what I got above and what equation 2.111 reproduced above gives?

  • $\begingroup$ What is the context here (just briefly)? $\endgroup$ Jun 7 '19 at 10:35
  • $\begingroup$ The goal of the relevant section in the book is to find the mean and covariance of a conditional Gaussian distribution p(x|y) where the joint is partitioned into x and y $\endgroup$ Jun 7 '19 at 10:38
  • $\begingroup$ You've some typos in $\mu_b$,$\mu_a$. $\endgroup$
    – gunes
    Jun 7 '19 at 15:39

Expanding your solution gives: $$\begin{align}E[x|y]&=\mu+(\Lambda+A^TLA)^{-1}(A^TL)(y-b)\\&-(\Lambda+A^TLA)^{-1}A^TLA\mu\end{align}$$ The second summand here is equal to the first expression in the answer, when expanded. We just need to show that sum of first and third summands is equal to the second expression in the answer: $$\begin{align}e&=\mu-(\Lambda+A^TLA)^{-1}A^TLA\mu=\mu-(\Lambda+A^TLA)^{-1}(A^TLA+\Lambda-\Lambda)\mu\\ &=\mu-(\Lambda+A^TLA)^{-1}(A^TLA+\Lambda)\mu+(\Lambda+A^TLA)^{-1}\Lambda\mu\\&=\mu-\mu+(\Lambda+A^TLA)^{-1}\Lambda\mu\\&=(\Lambda+A^TLA)^{-1}\Lambda\mu\end{align}$$ which is really the second expression in the answer. So, your result is equivalent to the given one.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.