3
$\begingroup$

I'm reading "Pattern recognition a machine learning" by Bishop (link). I need some mathematical help understanding ecuation 2.71. How do we obtain $\mu$ from "completing the square" on the exponent of a gaussian distribution (see the paragraph following equation 2.71).

Thank & Regards

$\endgroup$
2
$\begingroup$

The expression inside the exponential for a normal distribution is $-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu)$ which he rewrites as $-\frac{1}{2}x^T\Sigma^{-1}x+x^T\Sigma^{-1}\mu+const$. So whenever we have any expression inside the exponential of a normal distribution, we know that the term that is linear in $x$ is equal to $\Sigma^{-1}\mu$.

Take the example above in 2.70 where he breaks apart the parts of $x$ into $x_a$ and $x_b$. He wants to find the condition distribution of $x_a$ when you have $x_b$ as a given. So we can rewrite the standard equation in the form of 2.70, but how can we find $\mu_{a|b}$ or $\Sigma_{a|b}$ now that $x_b$ is a given constant?

Well, we know from before that for a normal distribution on $x$, the term that is linear in $x$ inside the exponential is equal to $\Sigma^{-1}\mu$. Bishop wants to find the distribution on $x_a$ given $x_b$. So he rewrites 2.70 with $x_b$ as a constant and finds the term that is linear in $x_a$ is equal to $x_a^T\{\Lambda_{aa}\mu_a-\Lambda_{ab}(x_b-\mu_b)\}$ (2.74). Our previous result tells us that term is equal to $\Sigma^{-1}_{a|b}\mu_{a|b}$. He follows the similar process in 2.72 and 2.73 to get $\Sigma^{-1}_{a|b}$ and substitutes that in to find $\mu_{a|b}$ in 2.75.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.