I understand naive Bayes is used largely in text classification. However, the number of features tend to outnumber the number of documents. Does this not result in overfitting where the number of parameters outweigh the number of samples.

I am trying to classify a set of documents using multinomial naive Bayes but however I have only achieved 80% accuracy using cross validations of 10 folds. Could overfitting be a problem in this?

I have about 25,000 features before features subset selection and after feature selection about 5000+ features. However, some of my samples set per class does not exceeds 1000 documents and I have about 7 classes.

  • $\begingroup$ If you're worried about it you could try reducing the dimensions of your documents by using a continuous space word model. $\endgroup$ – Aaron May 19 '14 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.