1
$\begingroup$

this is a very basic question about SVM.

I was using SVMs that are provided in the scikit for some problems, and noted that they are quite slow for big datasets. I then learned more about the implementation of the SVMs in the coursera course, and there the professor says that the SVM in practice often uses all the input vectors in the training set as the kernel. From my understanding it meant that there's as many "features" (I don't know if "feature" is correct word when SVMs are concerned) as there are input samples. Obviously, that would explain the slowness, because the training time is now quadratic in number of input samples.

However, I often see people on forums saying that they used SVMs in a context where it is very clear that the training set is quite large.

So my question is: is my understanding of how SVMs work completely wrong, and using all the input samples as the kernel does not make it quadratic, or do people use some other kernels often in practice? And if yes, what are the common kernels that can be used with SVMs?

$\endgroup$

1 Answer 1

1
$\begingroup$

Kernel SVM has a training complexity above quadratic in terms of the number of training samples. For low misclassification penalties (C) the complexity is close to quadratic, for high values it approaches cubic.

When you see people use SVM on very large datasets it is almost always linear SVM, which can be trained with sublinear complexity. For more details, please refer to LIBLINEAR and Pegasos. If you really want a nonlinear model, it is recommended to use ensemble models, for instance as is available in EnsembleSVM. The latter is mainly used when you have a lot of training instances (standard SVM is slow) in few dimensions (linear SVM is inaccurate).

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.