I have learned several classifiers in Machine learning - Decision tree, Neural network, SVM, Bayesian classifier, K-NN, Markov process...etc.

Can anyone please help to understand when I should prefer one of the classifier over other - for example - in which situation(nature of data sets, etc) I should prefer decision tree over neural net OR which situation SVM might work better than Bayesian OR what types of problems are appropriate to apply decision tree, neural net, k-nn, markov processs or SVM or bayesian ??


  • $\begingroup$ boosting with trees seems like its a good default classifier - non parametric + regularised model search. $\endgroup$ – probabilityislogic Jul 12 '13 at 7:13
  • $\begingroup$ The question is too broad. There are books written about it and even then it all comes down to so many factors, tricks and tweaks that it is hardly feasible to have a reasonable answer, no matter how long... $\endgroup$ – sashkello Jul 12 '13 at 13:21

There are tons of literature on the subject, but I have here a slightly subjective view that, I hope, does not duplicate other answers.

The first important thing -- performance of an ML algorithm is not the only criterion that one should consider. For me, the two other criteria are (i) simplicity and (ii) transparency, and arguably both of these are linked.

Simplicity means the simplicity of the algorithm used. This has many ramifications; first of all, in my subjective view, statistics in science is about convincing yourself and your peers that the effect seen is "real" (or, that it is unlikely to be random). If you have to rely on a magic box which works in mysterious ways (or you just don't understand how it works), you will not be able to trust its results.

Transparency means that you are able to discover why your ML model classifies the data like it does, select important variables based on the ML model (e.g. using loadings from PLS or Gini index from random forests), construct a simplified model etc.


Not the answer you're looking for? Browse other questions tagged or ask your own question.