Why does Naive Bayes outperform Support Vector Machines?

Question

I have a dataset composed of about 36000 attributes and 550 samples, the dataset is generated from text communication between people in some chatrooms.

The questions is when I try to classify these samples, a Naive Bayes classifier always outperforms a support vector machine, both in speed and accuracy. But in literature it is always noted that SVM is better in text-mining classification tasks.

Can anyone please explain in which situations Naive Bayes is better and in which situations SVM?

For more information about the question:

I am using the RapidMiner tool, I'm using 10 fold cross validation with stratified sampling. for Naive Bayes, the Laplacian correction is applied and for SVM I use a dot kernel and other parameters are all in their defaults, but when I change the parameters and try again, I get same result; Naive Bayes still outperforms SVM.

Answer 1

I disagree with the explanation given in the other answer. SVM, works well for large dimensional problems with relatively few instances because it is well regularized. In this case, I suspect the problem is not the tool but rather how it is being used.

For SVM I use a dot kernel and other parameters are all in their defaults ...

This is why your SVM results are bad. If you do not tune the SVM parameters (probably $c$ in your case), the resulting classifier will likely be poor unless you happen to get lucky with the default value.

when I change the parameters and try again, I get same result; Naive Bayes still outperforms SVM.

How do you change the parameters? What search method do you use? Do you just pick a value and hope for the best? Parameter search is important and must be done properly. Typically, optimal parameters are found through cross-validation.

Note that Naive Bayes may well be better for your particular application. Just because SVM is known to work well on this type of problems, does not mean that it always does.

Answer 2

Your dataset has a lot more attributes than instances (or samples). It is possible that the SVM doesn't have enough data to learn from. The Naive Bayes classifier can deal with such situations better. Due to the independence assumptions, the amount of instances it needs does not scale exponentially with the number of attributes, as is true of many other classifiers.