# Need new strategy for single class classifier

I am attempting to create a single class classifier where the classes are fairly close to balanced (+/- 25). My dataset has ~2,800 samples and ~1,100 features. All of the features are binary except for one (length of a document related to each record.) Some of the features are very sparse, ~650 of the features show up in only 5 or less samples. I've tried using a randomized search for random forest parameters, and after much tinkering the best classifier I could produce was:

clf = AdaBoostClassifier(sklearn.ensemble.RandomForestClassifier(n_estimators=1500, max_features = 4, criterion='gini', max_depth=None, bootstrap=True, random_state=42, class_weight='balanced'), algorithm="SAMME", n_estimators=200)


My cross validation results are:

           precision    recall  f1-score   support

0       0.64      0.55      0.59       374
1       0.57      0.66      0.61       344

avg / total   0.61      0.60      0.60       718


In this particular case I only care about classifying the 1's well, the precision and recall for the 0 class doesn't matter to me. I also care a lot more about precision than recall but I would ideally like to have at least 50% recall. I'm out of idea about what I can do to improve my results and would love to hear about some strategies that you may have.

Thank you!

Edit: I should probably mention that I'm fairly positive that the predictive power of my features is pretty low.

• This a nice paper that helped me when I was dealing with single classes: cs.uic.edu/~liub/publications/ICDM-03.pdf May 5 '16 at 14:16
• Keep in mind that if you keep adjusting your model by hand to optimize your cross-validation results, you run the risk of overfitting the cross-validated estimates. See web.archive.org/web/20120708051027/http://www.gregorypark.org/… for an example of this issue. May 5 '16 at 15:43
• Features that show up in only 5/2800 or less samples are called "features with low variance" and are typically removed. See, for example, scikit-learn documentation May 5 '16 at 18:38