I am attempting to create a single class classifier where the classes are fairly close to balanced (+/- 25). My dataset has ~2,800 samples and ~1,100 features. All of the features are binary except for one (length of a document related to each record.) Some of the features are very sparse, ~650 of the features show up in only 5 or less samples. I've tried using a randomized search for random forest parameters, and after much tinkering the best classifier I could produce was:
clf = AdaBoostClassifier(sklearn.ensemble.RandomForestClassifier(n_estimators=1500, max_features = 4, criterion='gini', max_depth=None, bootstrap=True, random_state=42, class_weight='balanced'), algorithm="SAMME", n_estimators=200)
My cross validation results are:
precision recall f1-score support 0 0.64 0.55 0.59 374 1 0.57 0.66 0.61 344 avg / total 0.61 0.60 0.60 718
In this particular case I only care about classifying the 1's well, the precision and recall for the 0 class doesn't matter to me. I also care a lot more about precision than recall but I would ideally like to have at least 50% recall. I'm out of idea about what I can do to improve my results and would love to hear about some strategies that you may have.
Edit: I should probably mention that I'm fairly positive that the predictive power of my features is pretty low.