I am working on a binary classification problem that is reasonably-sized (100k observations). I extracted 60 numerical features; the classes in the training set are well balanced. There are some significant linear patterns, but after that the patterns seem very nonrandom and so I need classifier models that can deal with this.
I am really looking to squeeze out the best possible (estimated) accuracy, at the sacrifice of computational effort, so I am considering creating an ensemble classifier.
So far, I have received pretty good results with:
- a Random Forest classifier (90% CV accuracy)
- a radial basis SVM classifier (87% CV accuracy, still busy tuning it on a finer grid).
I am now wondering if there are any other potentially interesting algorithms that I could add to the mix (three would be nice for majority voting, for example). I hope diverse models will help me shed some potentially remaining bias and improve accuracy a little bit. Preferably I'd use algorithms available through R's caret package. I am looking at Gaussian processes right now.
My background in machine learning is not very theoretical; I really only have intimate experience with SVM, decision trees and random forests, so the list of algorithms in caret is pretty daunting and I am having a hard time finding applied studies that compare them. I know it's hard to predict relative performance on particular data, but I'm willing to burn through a few of them!