I'm trying to build a classifier for my highly imbalanced binary data, and I'd appreciate some help on how to balance by results. The dataset has the following stats:
tabulate(classes) Value Count Percent 0 133412 97.62% 1 3247 2.38%
My dataset has 113 features. I'm using a boosting ensemble classifier with the RUSBoost algorithm (as my dataset is highly imbalanced). My weak learners are decision trees with a maximum of 5125 splits (1/16 of my training dataset examples). I'm using 300 learning cycles and a learn rate of 0.1. I get the following results (with 60% training and 40% testing):
accuracy: 0.99398 sensitivity: 0.87596 specificity: 0.99685 PPV: 0.87126 NPV: 0.99698
As can be appreciated, the classifier is getting very high specificity (and NPV), but not-so-good sensitivity (or PPV). Hence, my question is:
How can I change my classifier in order to get a balanced sensitivity and specificity (and of course PPV and NPV)? For example, the values indicated in the ROC curve would be awesome.
Any suggestion is very appreciated!