While I realize choosing the "right" algorithm can vary depending on the task at hand, I'm curious as to why Naive Bayes is quite often used for things like spam-classification or sentiment-analysis.

What's the giveaway in a dataset that screams: "Use Naive Bayes on me!"?

  • 3
    $\begingroup$ It's not dominant, it's just very simple and does a decent job. $\endgroup$ – Alex R. Apr 14 '17 at 17:47

I'm taking your word for Naive Bayes' popularity here as language processing isn't my specialty:

One reason NB is useful is the bias–variance tradeoff. Spam/sentiment type data are often noisy and usually high-dimensional (more predictors than samples, $n \ll p$). The naive assumption that predictors are independent of one another is a strong, high-bias, one.

By assuming independence of predictors we're saying that covariance matrix of our model only has non-zero entries on the diagonal. Since estimating covariance structure in $n \ll p$ situations is very hard indeed we are usually forced to put some constraints on the problem. The independence assumption is a particularly strong constraint that yields a highly interpretable model. The introduced bias may sufficiently reduce variance that you get better predictions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.