3
$\begingroup$

I am unable to clearly see the main differences between SVM & SMO. While I get the fact that SMO provides better algorithm for QP solvers but I see that when I use this in Weka on my MacBook it nearly took 12 hours for 46 features (≈40K feature vectors) of size 5MB dataset (whereas SVM took about 50 minutes).

Where does the optimization kick in? or Whats the catch. FYI, I am trying to build a binary classifier.

$\endgroup$
4
$\begingroup$

Sequential Minimal Optimization (SMO) is one way to solve the SVM training problem that is more efficient than standard QP solvers.

SMO uses heuristics to partition the training problem into smaller problems that can be solved analytically. Whether or not it works well depends largely on the assumptions behind the heuristics (working set selection). Typically, it speeds up training by quite a bit.

I'm not sure what Weka uses under the hood. LIBSVM is by far the most popular library to train SVMs, so you may want to look into that.

$\endgroup$
  • $\begingroup$ yes weka too uses java implementation of LibSVM. So... $\endgroup$ – Srikar Appalaraju Dec 26 '14 at 8:39
1
$\begingroup$

Whether training a SVM takes long or not depends on the data you are dealing with and the value of the parameters. Concretely, if the different classes overlap and you use a high value of C, then training will take long.

Also, if you have very skewed data, you are forced to use a higher value of C, which again leads you to a long training time. There is a solution for this problem though.

Lastly, do you pre-process your data? Normalizing your data can help reducing the training time. See the FAQ (previous link) and also this guide for advice on how to make use of SVMs.

$\endgroup$
  • $\begingroup$ thanks for the tip. I was not pre-processing my data. But still, SMO takes way more time than SVM. I believe my training data is not that big. It has a mix of categorical, numerical & float features. $\endgroup$ – Srikar Appalaraju Dec 26 '14 at 9:56
  • $\begingroup$ As Marc previously said, SMO is just an algorithm to solve a particular kind of QP problems. It would help us to understand your problem, if you tell us the different ways you fit the data. $\endgroup$ – jpmuc Dec 26 '14 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.