Document classification with Naive Bayes performs worse than other methods

Question

I am doing a document classification challenge on hackerrank.com. The training data is $X$ strings classified into $8$ classes. My approach is to use word frequencies and naive bayes (GaussianNB() from sklearn) instead, since I've read this is good for document classification. Specifically, I found the $Y$ most common words for each class, put this set $Y$ for each class into a large set of all common words. The features were then how many times each word in this large set occurred in the string.

The result of this for naive bayes classifier was a sklearn.metric.accuracy_score (on unseen data) of less than $0.75$ while other methods such as sklearn.svm.SVC and sklearn.tree.DecisionTreeClassifier achieved a $0.8$ to $0.9$. These scores were for choosing $Y$ (number of words from each class to use as features) to around 100. For $Y=10, 4$ etc the score was much lower.

Considering the cutoff accuracy is $0.95$, I am wondering what I have done wrong for two reasons:

1. Why are svm and decision tree performing better than naive-bayes, for a problem naive bayes is supposedly good at?
2. Is there something fundamentally wrong with my feature selection or choice of methods preventing me from achieving a higher score, or is it likely due to some error in the code?

Note that I have tried other features, like avg. word length, number of characters, with worse results for all methods.

Answer 1

Naive Bayes is often given as an example for algorytm that can be used for textual data, but this doesn't mean that it usually performs better. It certainly can be the case that for some dataset. Moreover, it often will be the case that other algorithms work better, because Naive Bayes makes very strong assumptions about conditional independence that are unlikely to hold.