# Feeding Naive Bayes classification model background statistics prior to fitting

I have a few thousand tagged documents that I use to train a Naive Bayes text classifier (using sklearn in this case). In addition to the tagged documents, I have about 100k untagged documents.

The accuracy is OK (~78%), but the model is overfitting the training data. I want to try to weigh the tfidf features after their frequency in the whole corpus before I feed the feature vector to the model. I'm sure there is a name for this procedure but I don't know what it is.

For instance, a specific word may be very rare in the training data but common in the untagged dataset. I want to take statistics such as that into account. How do I do that?

You should take a look at Transformed Weight Complement Naive Bayes.

https://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf