Conditional multivariate Gaussian distribution

Question

I am trying to understand how 'completing the square' method is used to determine mean and co-variance of multivariate Gaussian distribution but I am not able to make much sense of the following step $$−((x−μ)^TΣ^-(x − μ))/2 = −(x^TΣ^−x + x^TΣ^-μ + const)/2$$ where x and μ are vectors and Σ is co-variance matrix.

What does this step indicate and how is it derived?

P.S. I am not well versed with linear algebra which might be a reason for not being able to comprehend this fully, if you can point me to certain properties from linear algebra that are being used here, that will be very helpful

Answer 1

First, here's how you would complete the square with scalars (no matrices!)

$$ax^2 - 2bx = a\left(x-\frac{b}{a} \right)^2 + \frac{b^2}{a}$$

Please verify this for yourself.

Now, think intuitively about how you could rewrite this expression for matrices. Generally, a quadratic term $az^2$ for scalar values corresponds to a term $z^T A z$ when in matrix form. This is sometimes called a "quadratic form". Another thing is that we can't divide matrices, but we can multiply by the inverse, so a fraction like $\frac{b}{a}$ is usually converted to $A^{-1} b$.

Now we can rewrite the scalar complete-the-square equation in matrix form.

$$x^TA x - 2b^Tx = \left(x-A^{-1}b \right)^TA\left(x-A^{-1}b \right) + b^TA^{-1}b$$

Please expand this equation and verify that it is actually correct, and that the ad-hoc reasoning we did above actually works.

Now replace $b = \Sigma^{-1} \mu$ and $A = \Sigma$, and $\text{const} = b^TA^{-1} b$. You should arrive at something which resembles equation in the question.

Some linear algebra tips which might help: The matrix $A$ or $\Sigma$ is generally treated as symmetric without loss of generality, so $A = A^T$. Also I believe you wrote $\Sigma^-$ in your post but meant $\Sigma^{-1}$