In the Elements of Statistical learning book when introducing Linear Discriminant Analysis it says:

A simple application of Bayes theorem gives us

$Pr(G=K|X=x) = \frac{f_k(x)\pi_k}{\sum_{l=1}^Kf_l(x)\pi_l}$

where $\pi_k$ is the prior probability of class $k$ and $f_k(x)$ is the class conditional probability.

  1. What is the class conditional probability? Is it $Pr(X=x|G=K)$?
  2. How is derived the above equation from the Bayes theorem? I know $Pr(G=K|X=x) = \frac{Pr(X=x|G=K)Pr(G=K)}{Pr(X=x)}$

I know that $Pr(G=k)=\pi_k$ but I do not know how to derive the rest of the equation.


If $K$ is the number of classes, we can correct the formula as follows and use lowercase $k$ for referring to a specific class: $$P(G=k|X=x)=\frac{f_k(x)\pi(x)}{\sum_{l=1}^K f_l(x)\pi_l}$$

The class conditional probability is $P(X=x|G=k)$, or in case of non-discrete RVs class conditional density we use $f(X=x|G=k)$. The derivation follows from your note in (2): $$P(G=k|X=x)=\frac{\overbrace{P(X=x|G=k)}^{f_k(x)}\overbrace{P(G=k)}^{\pi_k}}{P(X=x)}$$ And, we can expand $P(X=x)$ via Total Probability Theorem: $$P(X=x)=\sum_{l=1}^K P(X=x|G=l)P(G=l)=\sum_{l=1}^Kf_l(x)\pi_l$$

  • $\begingroup$ RV stands for real-valued? $\endgroup$ Sep 11 '19 at 12:21
  • 1
    $\begingroup$ Random Variable $\endgroup$
    – gunes
    Sep 11 '19 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.