I have a text classification problem to solve. I need to classify a given sample of text into one of two classes A or B.
My training set has about 30% A and 70% B. This is my prior. Now, when I build the classifier, I get a confusion matrix like so
A 0.65 0.35 Actual Class B 0.20 0.80 A B Predicted class
One issue that I noticed is that when most of the features are zero, the training data classifies the sample as A. However, the classifier has a prior bias towards B so if I get an evaluation sample that has very few features, I will misclassify it as A.
I attempted to remedy this by biassing the classifier towards A. When I do this, My matrix improves. The False negatives and false positives both reduce to 0.18 which is an improvement.
What I want to know is whether this makes sense from a mathematical perspective.