Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I'm reading "Pattern recognition a machine learning" by Bishop (link). I need some mathematical help understanding ecuation 2.71. How do we obtain $\mu$ from "completing the square" on the exponent of a gaussian distribution (see the paragraph following equation 2.71).
Thank & Regards
The expression inside the exponential for a normal distribution is $-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu)$ which he rewrites as $-\frac{1}{2}x^T\Sigma^{-1}x+x^T\Sigma^{-1}\mu+const$. So whenever we have any expression inside the exponential of a normal distribution, we know that the term that is linear in $x$ is equal to $\Sigma^{-1}\mu$.
Take the example above in 2.70 where he breaks apart the parts of $x$ into $x_a$ and $x_b$. He wants to find the condition distribution of $x_a$ when you have $x_b$ as a given. So we can rewrite the standard equation in the form of 2.70, but how can we find $\mu_{a|b}$ or $\Sigma_{a|b}$ now that $x_b$ is a given constant?
Well, we know from before that for a normal distribution on $x$, the term that is linear in $x$ inside the exponential is equal to $\Sigma^{-1}\mu$. Bishop wants to find the distribution on $x_a$ given $x_b$. So he rewrites 2.70 with $x_b$ as a constant and finds the term that is linear in $x_a$ is equal to $x_a^T\{\Lambda_{aa}\mu_a-\Lambda_{ab}(x_b-\mu_b)\}$ (2.74). Our previous result tells us that term is equal to $\Sigma^{-1}_{a|b}\mu_{a|b}$. He follows the similar process in 2.72 and 2.73 to get $\Sigma^{-1}_{a|b}$ and substitutes that in to find $\mu_{a|b}$ in 2.75.
