# Naive Bayes text classification on different cardinality classes

I have written a Naive Bayes classifier (with Laplace smoothing) and am using it to classify text into a few simple classes. However, I found that the classes are not of the same vocab size -- to be clear this is not the same as a problem with some classes being represented more often in the training data, since this would be addressed by the a priori probabilities or by resampling to equalize classes but neither of these helps on the different vocab sizes.

In short, my catch-all class "none" has a much wider vocabulary so P(w|c) is smaller on average for a given common word than in one of the other classes. Another way to see this is that the probability mass (even though Laplace smoothing gives some mass to every word) is much more concentrated in common words for my actual classes and much more spread out for my catch-all "none" class. This means that the more words there are, the more likely the class chosen will not be "none" even when it should be. How can I address this problem?

• Just to understand it better: you're classifying words, not texts, is is right? Because if you are just classifying words, it is not a problem to have a small $P(w|none)$, you just need it to be bigger than $P(w|c)$ for other classes $c$. Feb 4, 2015 at 13:55
• Sorry if I wasn't clear in my question, I am classifying texts and so having smaller probabilities on average for common words when conditioning on the none class means that any other class becomes more likely to be the chosen one as the text gets longer. Since asking the question, to address this I used a stop words list and did some tf-idf weighing, which seems to do some good work towards this but I was wondering what the correct and more general way to solve this would be. Thanks! Feb 4, 2015 at 22:41