# SVM training method (or alg.)

I'm using SVM classification (Matlab) within my research works, and I want to know:

1. The advantages and disadvantages of each training algorithm, i.e., SMO, LS and QP
2. In general case, what is the suitable algorithm?
3. If the choice of one of them can impact the classification performance?
4. If the are any relationship between the choice of training method (or algorithm) and the choice of the kernel function.

1) The advantages and disadvantages of each training alg. i.e SMO, LS and QP

SMO: state-of-the-art for nonlinear SVM. Always use SMO over traditional QP as it is way faster. By LS I assume you refer to LS-SVM, which is a distinct algorithm from standard SVM and should not be compared directly.

2) In general case, what is the suitable alg. ?

SMO.

3) If the choice of one of them can impact the classification performance ?

No. Unless you heavily use additonal heuristics to decrease training time.

4) If the are any relationship between the choice of training method (or alg.) and the choice of the kernel function.

No. The choice of kernel defines the optimization problem. The methods you list are ways to solve said optimization problem (SMO and QP solve traditional SVM, LS-SVM is a different optimization problem).

• Thank you for this response. But precisely, I would like to know the main difference(s) between SVM (SMO) and LS-SVM. The most performant in general case and the faster. Commented Dec 17, 2013 at 14:20
• Performance-wise they are competitive. Nonlinear LS-SVM trains faster than nonlinear SVM (solving linear system vs. solving QP). Predicting with an SVM is faster than LS-SVM because SVM is sparse whereas LS-SVM is not. Commented Dec 17, 2013 at 14:21
• Thank you. But what did you mean by "because SVM is sparse"? Commented Dec 17, 2013 at 14:28
• An SVM model is based on the so-called support vectors, which is a subset of the training data (often a small fraction, hence sparse). In an LS-SVM model, all data instances become support vectors unless some additional measures are taken. Commented Dec 17, 2013 at 14:34
• So can we said that LS-SVM is more efficient ? Commented Dec 17, 2013 at 15:43