2
$\begingroup$

I'm creating a naive bayes text classifier, but I'm wondering if it's a good idea to break the text up into both unigrams and bigrams. Should I only use one method? Will having both variations mess with the algorithm?

$\endgroup$
1
$\begingroup$

Having a variation is a very common practice. It actually has a smoothing effect.

Bigram --> Low bias/high variance Unigram --> High bias/low variance.

Combination the two helps to "hedge" the bets made between the two. See for instance, the first equation on pg. 13 of this link. The author shows how to merge trigram, bigram and unigram estimates

$\endgroup$
0
$\begingroup$

Starting simple with just unigrams and then just bigrams would likely be ideal. If you use both then the complexity of the model would increase, and it might not even result in an increase in performance. If you do include both unigrams and bigrams, and that results in a better classifier, I don't see why not!

$\endgroup$
  • $\begingroup$ By combine, do you mean into one classification or having two separate classifications and then deciding the outcome based on the ratings for both bigrams and unigrams? It just dawned on me that I could actually do that... $\endgroup$ – photocode Jan 25 '17 at 20:11
  • $\begingroup$ I guess you could do either of those options. It might be easier to include into one model, but you run into some serious correlations between unigrams and bigrams, which goes against the "Naive" part of Naive Bayes. That might result in some funky behaviour, but you could definitely assess that with how good your classifier is working after trying it out. I would still suggest starting simple with one or the other, and only increasing in complexity if it's not working very well. $\endgroup$ – smccain Jan 25 '17 at 21:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.