I currently have a problem where I am trying to classify medical abstracts where some are relevant and some aren't. I have tried an SVM, Multinomial Naive Bayes and Random Forest, and found the MNB works best for the task at hand. The only issue is, my recall for the "relevant" class is slightly lower than it should be. For the datasets I've tested on, my recall is between 0.6-0.85.
Ideally I want the recall for this class to be 0.95+, even if it comes with a decrease in overall classification rate. I would like to minimize the number of False Negatives as much as possible, even if that means the number of False Positives shoot up.
scikit-learn to implement my models. I tried running a grid search with different alphas and my own scoring function (recall for the class I want) but it just returned the same parameters I was using before.
Any help would be greatly appreciated.