13
$\begingroup$

SVM algorithm is quite old - it was developed 1960s, but was extremely popular in 1990s and 2000s. It is a classical (and quite beautiful) part of machine learning courses.

Today it seems that in media processing (images, sound etc.) neural networks completely dominate, while in other areas Gradient Boosting has very strong positions.

Also, in recent data competitions I observe no SVM-based solutions.

I am looking for application examples where SVM still gives state-of-art results (as of 2016).

Update: I'd like to have some example which I can give e.g. to students / colleagues when explaining SVM so that it doesn't look like purely theoretical or deprecated approach.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Superior in what sense? Some performance metric? Training a deep neural net requires a substantial amount of computer time, but I can train a serviceable SVM on my laptop. $\endgroup$
    – Sycorax ♦
    Commented Jan 21, 2016 at 3:20
  • $\begingroup$ @user777 I mean classification / regression metric appropriate for the application field, of course. The issue with computational complexity for DL is important, but this is a bit out of the scope of this question. $\endgroup$
    – Alleo
    Commented Jan 21, 2016 at 3:25

1 Answer 1

Reset to default
12
$\begingroup$

According to the paper Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? SVM together with Random Forest and Gradient Booting Machines are among the top performing classification algorithms for a large set of 120+ datasets (using accuracy as metric).

I repeated their experiments with some modifications and I get these three classifiers performing better than the others, but as the no free lunch theorem says there are always a problem where some other algorithm performs better than these three.

So yes, I would say that SVM (with Gaussian kernel - that is what I used) is still a relevant algorithm for non-media related datasets.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi, thanks for response! I've seen this interesting study. As far as I understand, the idea was to see how much classifier gives without any serious tuning (while data analyst should perform tuning IMO). An area-related study would be of more interest. $\endgroup$
    – Alleo
    Commented Jan 21, 2016 at 3:55
  • 1
    $\begingroup$ I remember that Delgado et all did not perform a very detailed search for the best hyperparameters, but if they did perform some search. The question (for which I have no answer) is whether a more fine grained search for the best hypeparameters would result in different results. If that is true it would mean that the competing algorithms to SVM have in general a very sharp peak in the accuracy for particular hyperparameters, which I think is a negative factor for the algorithm. $\endgroup$
    – Jacques Wainer
    Commented Jan 21, 2016 at 4:13
  • $\begingroup$ Also one minor comment is that UCI datasets (used for tests) mostly quite small. I wonder if this could be the explanation to poor results of boosting? Most of kaggle challenges (with much data) demonstrate the superior performance of GB. $\endgroup$
    – Alleo
    Commented Jan 21, 2016 at 13:28
  • $\begingroup$ I agree the data sets are small. For larger datasets I have been using Random Forests at the moment - will start using GBM as soon as I am more comfortable with the hyperparameters - I dont know how sensible is GBM to them. $\endgroup$
    – Jacques Wainer
    Commented Jan 21, 2016 at 16:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.