I am attempting to create a single class classifier where the classes are fairly close to balanced (+/- 25). My dataset has ~2,800 samples and ~1,100 features. All of the features are binary except for one (length of a document related to each record.) Some of the features are very sparse, ~650 of the features show up in only 5 or less samples. I've tried using a randomized search for random forest parameters, and after much tinkering the best classifier I could produce was:

clf = AdaBoostClassifier(sklearn.ensemble.RandomForestClassifier(n_estimators=1500, max_features = 4, criterion='gini', max_depth=None, bootstrap=True, random_state=42, class_weight='balanced'), algorithm="SAMME", n_estimators=200)

My cross validation results are:

           precision    recall  f1-score   support

      0       0.64      0.55      0.59       374
      1       0.57      0.66      0.61       344

avg / total   0.61      0.60      0.60       718

In this particular case I only care about classifying the 1's well, the precision and recall for the 0 class doesn't matter to me. I also care a lot more about precision than recall but I would ideally like to have at least 50% recall. I'm out of idea about what I can do to improve my results and would love to hear about some strategies that you may have.

Thank you!

Edit: I should probably mention that I'm fairly positive that the predictive power of my features is pretty low.


Try increasing the weights of class 1 while training... This will ensure to make your algorithm to be extra careful with class 1. Also try different types of algorithms and tune them carefully with your metrics you care about.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.