In the Elements of Statistical learning book when introducing Linear Discriminant Analysis it says:

A simple application of Bayes theorem gives us

$Pr(G=K|X=x) = \frac{f_k(x)\pi_k}{\sum_{l=1}^Kf_l(x)\pi_l}$

where $\pi_k$ is the prior probability of class $k$ and $f_k(x)$ is the class conditional probability.

  1. What is the class conditional probability? Is it $Pr(X=x|G=K)$?
  2. How is derived the above equation from the Bayes theorem? I know $Pr(G=K|X=x) = \frac{Pr(X=x|G=K)Pr(G=K)}{Pr(X=x)}$

I know that $Pr(G=k)=\pi_k$ but I do not know how to derive the rest of the equation.


1 Answer 1


If $K$ is the number of classes, we can correct the formula as follows and use lowercase $k$ for referring to a specific class: $$P(G=k|X=x)=\frac{f_k(x)\pi(x)}{\sum_{l=1}^K f_l(x)\pi_l}$$

The class conditional probability is $P(X=x|G=k)$, or in case of non-discrete RVs class conditional density we use $f(X=x|G=k)$. The derivation follows from your note in (2): $$P(G=k|X=x)=\frac{\overbrace{P(X=x|G=k)}^{f_k(x)}\overbrace{P(G=k)}^{\pi_k}}{P(X=x)}$$ And, we can expand $P(X=x)$ via Total Probability Theorem: $$P(X=x)=\sum_{l=1}^K P(X=x|G=l)P(G=l)=\sum_{l=1}^Kf_l(x)\pi_l$$

  • $\begingroup$ RV stands for real-valued? $\endgroup$ Commented Sep 11, 2019 at 12:21
  • 1
    $\begingroup$ Random Variable $\endgroup$
    – gunes
    Commented Sep 11, 2019 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.