If I want to use naive bayes with laplace smoothing and therefore add 1 to probabilities with the value of 0, what does this mean for probabilities which have the actual value of 1?

  • $\begingroup$ It is not clear what you are asking. $\endgroup$ Apr 9 '17 at 22:46
  • $\begingroup$ When using laplace smoothing with naive bayes, do I have to add the value 1 to all my probabilities, or just probabilities with the value of 0? $\endgroup$
    – link
    Apr 9 '17 at 22:50
  • $\begingroup$ Yes I read your question but it doesn't make sense as written. $\endgroup$ Apr 9 '17 at 22:52
  • $\begingroup$ link ; perhaps the value is added to zero cells of counts (or all cells?) which are then used to estimate the probabilities, rather than adding a value to the probabilities themselves $\endgroup$
    – user20650
    Apr 9 '17 at 22:59
  • $\begingroup$ perhaps of interest stats.stackexchange.com/questions/108797/… $\endgroup$
    – user20650
    Apr 9 '17 at 23:03

A value 1 is added to each feature count (not just to the feature having a count/frequency of 0 ). To be more specific, consider the case of text classification. Let us suppose that $Y_k, k=1,2,\cdots K$ denote the labels of the $K$ classes, $X_j$ denote the $j^{th}$ word and $V$ denote the total number of distinct words(vocabulary size) in all of the $n=\sum_{k=1}^{K}n_{k}$ documents, where $n_{k}$ denote the number of documents labelled as $Y_{k}$. Then, the Laplace estimate of the probability for the word $X_{j}$ in the class $Y_{k}$ is given by $$P(X_{j}|Y_{k})\dfrac{Count(X_{j},Y_{k})+1}{\sum_{j}^{V}(Count(X_{j},Y_{k})+1)}.$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.