Machine Learning Methods for Binary Classification I was hoping to get a nice list of alternatives to logistic regression and decision trees for binary classification ("Yes vs. No" or "Cured vs. Not cured"). I am more interested in identifying the variables associated with the outcome than finding the model which predicts the outcome best. 
Obviously logistic regression is a great, simple way to do this. I have used classification trees as well. It would be great to learn about other methods. Could support vector machines be used for this task as well?
 A: I don't really believe svm's would fit here as the features used in this classifier are numerical and hence, are not easy to be interpreted. What about Naive Bayes? It can serve as a binary classifier and at the same time you can "see" which features contribute to the class decision...
What type of features do you have? nominal, ordinal, numerical, ...?
A: This is more or less what I did for my masters thesis. I used elastic net regression and it seemed to work well, although I didn't do any sensitivity analysis. Other penalties (lasso, probably even SCAD) would work as well but elastic net was best for my use.
The question also comes down to what you mean by "important." Elastic net or lasso regression, on mean-centered and standard-deviation-scaled data, will select individual features with the largest effect on the outcome.
If, instead, you are interested in the individual variables that decrease predictive accuracy the most when removed from the full model, then this method won't be appropriate. Here you would fit any classifier, then for each predictor re-fit the classifier without that predictor and see how much the predictive accuracy falls.
You can also use regression here but remember that regression is not a true classifier. Rather, it estimates expected probabilities of class membership. If you want or need a true classifier you should use an SVM or decision tree, or better still an ensemble method like AdaBoost or Random Forests.
