Discriminant function for diagonal LDA

Question

I have a question on diagonal LDA in the book "Elements of statistical learning" by Hastie, Tibshirani et al. On page 652, it says

The discriminant score of the diagonal-covariance LDA for class $k$ is $$ \delta_{k}(x)= - \sum_{j=1}^{p} \dfrac{(x_{j}-\bar{x}_{kj})^{2}}{s_{j}^{2}} + 2 \log \pi_{k}, $$ [see (4.12) on page 110].

First thing I'm wondering about is the fact that (4.12) is the discriminant function of QDA and not of LDA, I thought diagonal LDA is based on LDA?

Nevertheless, this is (4.12):

$$ \delta_{k}(x) = - \frac{1}{2} \log |\mathbf{\Sigma}_k| - \frac{1}{2}(x-\mu_k)^T\mathbf{\Sigma}_k^{-1}(x-\mu_k)+\log \pi_k $$

$\mathbf{\Sigma}_k$ is the covariance matrix of each class $k$. For the transformation below I used $\mathbf{\Sigma}_k = \mathrm{diag}(\sigma_j^2) \; \forall k$.

So now I would like to justify why dLDA has the above discriminant function. With what I know, I was able to transform (4.12) into

$$\delta_{k}(x) = - \frac{1}{2} \log |\mathrm{diag}(\sigma_j^2)| - \frac{1}{2} \sum\limits_{j=1}^p\frac{(x_j-\mu_k)^2}{s_j^2}+\log \pi_k. $$

Explanation of the variables:

• $s_j$ is the Standard Deviation of the $j$th gene
• $\bar{x}_{kj}=\sum_{i \in C_{k}}\frac{x_{ij}}{N_{k}} $ mean of the $N_{k}$ values for gene $j$ in class $k$
• $C_{k}$ Indexset for class $k$
• $\pi_k$ Prior probability of class $k$, $\sum_{k=1}^{K}\pi_{k}=1$
• I think $\mu_k$ = $\bar{x}_{kj}$

So now I would really appreciate some hints how to proceed in order to transform some more so I end up with the discriminant function of dLDA.

Answer 1

I could solve my problem thanks to amoeba's comments.

My first Transformation got me to:

$\delta_{k}(x) = - \frac{1}{2} \log |diag(\sigma_j^2)| - \frac{1}{2} \sum\limits_{j=1}^{p} \dfrac{(x_{j}-\mu_k)^{2}}{\sigma_{j}^{2}}+\log \pi_k $

Because $\mu_k$ is the vector $(\bar{x}_{k1},...,\bar{x}_{kp})^T$ with $\bar{x}_{kj}$ being the mean of the $j$th gene in class $k$, it actually should have been $\delta_{k}(x) = - \frac{1}{2} \log |diag(\sigma_j^2)| - \frac{1}{2} \sum\limits_{j=1}^{p} \dfrac{(x_{j}-\bar{x}_{kj})^{2}}{\sigma_{j}^{2}}+\log \pi_k $.

The term $- \frac{1}{2} \log |diag(\sigma_j^2)|$ means $- \frac{1}{2} \log det(diag(\sigma_j^2))$ which is a constant. As the covariance matrices of all classes are equal, this constant is the same for all classes. $\delta_k(x)$ is used to compute the discriminant score of each class to then compare them to each other. Hence this constant can be dropped because it changes all scores equally.

For the same reason, the remaining $\delta_{k}(x) = - \frac{1}{2} \sum\limits_{j=1}^{p} \dfrac{(x_{j}-\bar{x}_{kj})^{2}}{\sigma_{j}^{2}}+\log \pi_k $ can be multiplied by $2$ and this equals the discriminant function in the book.