I am following CS229 course by Andrew Ng. On this lecture note it talks about using Laplace smoothing to bypass situations of 0-probabilities. What does not make sense is the immediate jump to the following line.
I cannot understand how this derivation is made. Can anyone show me the full derivation of how the factor of 2 comes into play? intuitively I feel that 2 is because it is a binary classifier, but a more mathematical reasoning is appreciated.
To avoid $\frac{0}{0}$ situation, we add one to each observed events, this is also called psuedo-counting . Since you increase the count of all possible events by 1, you have to increase the total number of trials by the number of possible events.
In binary Naive Bayes, since we increase each event(item being from 0 or 1 ) by 1 you have to change denominator to $N + 1 \times 2$.
In general, we denote $\alpha > 0$ as smoothing(psuedocounting) factor. THen your smoothed probability becomes, \begin{equation} Pr_{smoothed}(y=i | x) = \frac{1_{y=i} + \alpha}{ N + \alpha \times d} \end{equation} where $d$ is number of possible outcomes, and $i$ is a possible outcome out of $d$ outcomes. Hope this helps
