I know this question is not very hard to answer, but it posed as a problem for me, so it may pose as a problem to other people too.

We know that Bayes' Theorem is stated using the following formula:
$$Pr(A\big | B) = \frac{Pr(B \big | A)Pr(A)}{Pr(B)}$$ Where $A$ and $B$ are events and $Pr(B) \neq 0$. Now, in Introduction to Statistical Learning by James et al. it is given that

Bayes' Theorem states that $$Pr(Y=k | X=x) = \frac{\pi_kf_k(x)}{\sum_{l=1}^K\pi_lf_l(x)}$$

Where $\pi_k$ is the fraction of the training observations that belong to the $k$ th class, and $f_k(x)$ is the density function which is equivalent to $Pr(X=x|Y=k)$.
How are the two formulas related?


As given, $\pi_k$ is simply the probability that $Y=k$. And it is already given that $f_k(x)=Pr(X=x|Y=k)$. So, the numerators of both the formulas correspond to each other correctly. Now here comes the denominator: (let $K$ denote the total number of classes that $Y$ can take) $$ \sum_{l=1}^K\pi_lf_l(x) = \sum_{l=1}^KPr(Y=l)\times Pr(X=x|Y=l) \\ = \sum_{l=1}^KPr(X=x)\times Pr(Y=l|X=x) \\ = Pr(X=x)\times\sum_{l=1}^KPr(Y=l|X=x) \\ = Pr(X=x)\times 1 \\ = Pr(X=x)$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.