# Diagonal linear discriminant analysis

On p110 of Murphy's machine learning book, he derives the discriminant function for the diagonal LDA model by simplifying the full linear discriminant analysis equation (4.33):

It seems that he's discarded the normalization constants to get the expression for $\delta_c {\bf{x}}$. Why?

I haven't come across the notation $\delta_c {\bf{x}}$ before - what does this mean?

He's used the formula for conditional probability. Recall the conditional probability rule: $$p(A,B|C)=p(A|B,C)p(B|C)$$ This basically asserts that the joint equals the conditional times the normalizing ''constant''. The normalizing constant is given by the rule of marginalization: $$p(B|C)=\sum_a p(A=a,B|C)$$
(4.33) is basically saying: $$p(y=c|x,\theta)=\dfrac{p(x,y=c|\theta)}{\sum_cp(x,y=c|\theta)}=\dfrac{p(x,y=c|\theta)}{\color{red}{p(x|\theta)}}$$
The denominator in (4.33), which is the normalizing constant, is $p(x|\theta)$. $$\delta_c(x) =\log \underbrace{p(x,y=c|\theta)}_{\text{joint}}=\log [\underbrace{p(y=c|x,\theta)}_{(4.33)} \underbrace{p(x|\theta)}_{\text{(constant)}}]$$
$\delta_c(x)$ appears to be a fancy notation for the following: for each class $c$, determine $\delta_c(x)$ for a fixed $x$. This tells you to what extent $x$ belongs to each class, $c$.
• thanks. What about the $|2 \pi \Sigma_c |$ factors? Feb 5 '16 at 8:58
• In the formulation, all $\Sigma_c=\Sigma$. So the determinant factors cancel out. Feb 23 '16 at 5:52