I have a few thousand tagged documents that I use to train a Naive Bayes text classifier (using sklearn in this case). In addition to the tagged documents, I have about 100k untagged documents.
The accuracy is OK (~78%), but the model is overfitting the training data. I want to try to weigh the tfidf features after their frequency in the whole corpus before I feed the feature vector to the model. I'm sure there is a name for this procedure but I don't know what it is.
For instance, a specific word may be very rare in the training data but common in the untagged dataset. I want to take statistics such as that into account. How do I do that?
