Linear Discriminant Analysis for $p=1$

Question

I'm studying 'Introduction to Statistical Learning' by James, Witten, Hastie, Tibshirani.

In page 139, of their book, they began by introducing Bayes' Theorem $p_k(X)=P(Y=k|X=x) = \dfrac{\pi_kf_k(x)}{\sum_{l=1}^k \pi_l f_l(x)}$. $\pi$ is not mathematical constant, but denotes the prior probability. Nothing is strange in this equation.

The book claims that it wants to obtain an estimate for $f_k(x)$ that it can plug into the equation given above. To estimate $f_k(x)$, it assumes that is normal. In the one dimensional setting, $f_k(x)=\dfrac{1}{\sqrt{2\pi}\sigma}\exp(-\dfrac{1}{2\sigma^2}(x-\mu_k)^2)$, where $\mu_k$ and $\sigma^2_k$ are the mean and variance for the $k$th class. It is assumed that $\sigma^2_1 = \sigma^2_2 = \cdots = \sigma^2_K$. (I began to get confused from the last statement.)

Plugging $f_k$ into $p_x$, you have this quite messy equation (1) :

$$p_x(k)=\dfrac{\pi_k \frac{1}{\sqrt{2\pi}\sigma}\exp(-\frac{1}{2\sigma^2}(x-\mu_k)^2)}{\sum_{l=1}^K \pi_l \frac{1}{\sqrt{2\pi}\sigma}\exp(-\frac{1}{2\sigma^2}(x-\mu_l)^2)}. $$

Again, no surprises here since it is just substitution.

---

The Bayes' Classifier involves assigning an observation to the class for which equation (1) is the largest. Taking the log of Equation (1) and rearranging terms, it is not hard to show that this is equivalent to assigning the observation to the class for which the following is the largest:

$$\delta_k(x)=x \cdot \dfrac{\mu_k}{\sigma^2} - \dfrac{\mu_k^2}{2\sigma^2} + \log(\pi_k) $$

Question: I do not understand where this came from, and what it means. I tried doing the log of equation and it doesn't become this. Are we taking the derivative somewhere here, since this is the largest observation?

Answer 1

You can express equation (1) up to a proportionality constant,

$$p_x(k)\propto \pi_k \frac{1}{\sqrt{2\pi}\sigma}\exp \left(-\frac{1}{2\sigma^2}\left(x-\mu_k \right)^2 \right) $$

so if you then take logs

$$ \log p_x(k) \propto \log \pi_k - \log (\sqrt{2\pi} \sigma) -\frac{1}{2\sigma^2}\left(x-\mu_k \right)^2 $$

where $- \log (\sqrt{2\pi} \sigma)$ again goes into the proportionality constant because it doesn't depend on $k$. Then expand the squared term and you are there (note that expanding the bracket will give another term which will vanish into $\propto$).