SVM Vs Neural Network Vs Random Forest classifier comparison on multi class problem Any idea if SVM or Neural Net or Random Forest works better on a classification problem on the same multi class dataset?
I mean, in general, which should outperform the comparison?
 A: Every of the mentioned classifiers will be best on some datasets and some problems. No free lunch
A: Tree methods are good if;
1. You have lots of predictor variables and data.
2. Your test set values are within the range of values found within the
    training set (for both predictors and response). Trees tend to suck
    at extrapolative predictions although they have some success with
    panel data.
3. Suitable predictor variables are de-noised (e.g. wavelet
    transformation, etc).
SVM are likely better than tree methods if;
1. The dataset is smaller with less predictor variables.
2. It's a time series.
3. Extrapolative predictions are needed 
   (opposed to interpolative predictions).
All of this is subjective, there will be many datasets which like contradict everything I just said.
A: Ok I can answer this one in the case of SVM vs NN. I havn't looked into binary decision trees. 
Question: Most Accurate Overall
Answer: SVM
very important: There are going to be cases that NN will be better. But overall when it comes to classification problems SVM will out perform a NN. Assuming you are using a kernel (like RBF or sigmoid)
Why is that?
Neural Networks are great in generalising. It can compute non-linearity exceptionally well. Although you can get good estimations for non-linearity for SVM, it just can't compare to NN.
However, NN has its downside, especially in practical applications with non ideal data. The problem is that NN do need allot of data to be trained and configured properly, where SVM is much better in that regard.
NN can get stuck in local optima, where as in SVM the global Optima will ALWAYS be guaranteed.
SVM generally performs better with higher dimensional data, as when performing dimensionality reduction it is important to choose the desired dimensions for Neural Nets, however, this is not necessary for SVM.
Question: Best Performance Overall
Answer: Neural Networks
These two are about the same with SVM being a little slower. In truth there is no real difference... HOWVER, where Neural Networks is allot better in speed and performance is when you apply MapReduce to it. It is very easy to compute a NN in parallel, where it is early (but not) impossible to do SVM in parallel.
Final Note
Binary Trees seem to getting more popular, I would not disregard them yet. But you can layer SVM with a Binary Tree to make it even better. And Parallel functions for Binary trees is laughably easy.
Also I have only been doing machine learning for a short while so if someone can point out any inconsistencies with what is written above, I will be very thankful.
