# Introduction to Statistical Learning Eq. 4.32

Can someone please explain how the third line becomes the fourth line?

• For vectors $u$ and $v$ and a square symmetric matrix $A$ you ask whether $y^\prime A y + u^\prime A u = (y-u)^\prime A (y-u) + 2u^\prime A y.$ This algebraic identify follows from the two properties listed at the left along with the symmetry of $A.$ Perhaps seeing the relation in this simpler notation will help you work through the steps.
– whuber
Commented Feb 27, 2022 at 17:31

It is an issue of expanding and tidying up. You have for example

• $$(x-\mu_k)^T\Sigma^{-1}(x-\mu_k) = x^T\Sigma^{-1}x - \mu_k^T\Sigma^{-1}x-x^T\Sigma^{-1}\mu_k+\mu_k^T\Sigma^{-1}\mu_k$$ and
• $$\mu_k^T\Sigma^{-1}x=x^T\Sigma^{-1}\mu_k$$ and $$\mu_k^T\Sigma^{-1}\mu_K=\mu_K^T\Sigma^{-1}\mu_k$$ and
• $$(\mu_k+\mu_K)^T\Sigma^{-1}(\mu_k-\mu_K) = \mu_k^T\Sigma^{-1}\mu_k -\mu_K^T\Sigma^{-1}\mu_K$$

so

$$\log\left(\frac{\pi_k}{\pi_K}\right) -\frac12(x-\mu_k)^T\Sigma^{-1}(x-\mu_k) +\frac12(x-\mu_K)^T\Sigma^{-1}(x-\mu_K)$$
$$= \log\left(\frac{\pi_k}{\pi_K}\right) -\frac12 x^T\Sigma^{-1}x+ x^T\Sigma^{-1}\mu_k- \frac12\mu_k^T\Sigma^{-1}\mu_k +\frac12 x^T\Sigma^{-1}x- x^T\Sigma^{-1}\mu_K+ \frac12\mu_K^T\Sigma^{-1}\mu_K$$
$$= \log\left(\frac{\pi_k}{\pi_K}\right) - \frac12(\mu_k^T\Sigma^{-1}\mu_k - \mu_K^T\Sigma^{-1}\mu_K) + x^T\Sigma^{-1}\mu_k- x^T\Sigma^{-1}\mu_K$$ $$= \log\left(\frac{\pi_k}{\pi_K}\right) - \frac12\mu_k^T\Sigma^{-1}\mu_k -\mu_K^T\Sigma^{-1}\mu_K + x^T\Sigma^{-1}(\mu_k- \mu_K)$$

Another way would be adding some terms to make the first multiplicands the same, and group (subtract the added terms outside of the parantheses as well): \begin{align}A&=-\frac{1}{2}(x-\mu_k-\overbrace{\mu_K}^{new})^T\Sigma^{-1}(x-\mu_k)+\frac{1}{2}(x-\mu_K-\overbrace{\mu_k}^{new})^T\Sigma^{-1}(x-\mu_K)\\&- \frac{1}{2}\mu_K^T\Sigma^{-1}(x-\mu_k)+\frac{1}{2}\mu_k^T\Sigma^{-1}(x-\mu_K)\rightarrow \text{subtract newly added terms}\\&=\frac{1}{2}(x-\mu_k-\mu_K)^T\Sigma^{-1}(\mu_k-\mu_K)+\frac{1}{2}x^T\Sigma^{-1}(\mu_k-\mu_K)\\&=x^T\Sigma^{-1}(\mu_k-\mu_K)-\frac{1}{2}(\mu_k+\mu_K)^T\Sigma^{-1}(\mu_k-\mu_K)\end{align}

Note that $$x^T\Sigma^{-1}\mu=\mu^T\Sigma^{-1}x$$ since inverse of cov. matrix is symmetric.

• +1 for noting that symmetry is essential.
– whuber
Commented Feb 27, 2022 at 17:31