0
$\begingroup$

Hayy all,, Im going to do classification using SVM.
As I understand we have to project our data into higher dimensional by using kernel. And there are 4 common use kernel (linear, RBF, polynomial, and sigmoid).

  1. What is the different between those kernel?
  2. We need to find the linear separable line / hyperplane to classify our data. And to do that, we have to chose large margin to avoid any overfitting? What is overfitting?
  3. As u can see in this image, there is a w. And in many reference, they said w is a line? What is that actually? And what is that lamda??
  4. What is SMO mean in SVM?
$\endgroup$
  • 1
    $\begingroup$ Oh, so I can make many thread to ask my question separately?? $\endgroup$ – user3507478 Apr 9 '14 at 18:02
1
$\begingroup$

What is the different between those kernel?

There are several differences. The linear kernel, for example, effectively works in input space while the others do not. The sigmoid kernel should be avoided for numerical reasons (it is basically inherited from neural networks).

We need to find the linear separable line / hyperplane to classify our data. And to do that, we have to chose large margin to avoid any overfitting? What is overfitting?

Overfitting means your model is fitting noise in the data instead of the underlying process. Such models have poor generalization performance (e.g. on unseen data). This is symptomized by high training accuracy but low test accuracy.

As u can see in this image, there is a w. And in many reference, they said w is a line?

$\mathbf{w}$ is the separating hyperplane in feature space.

And what is that lamda??

You forgot to show an equation, so it's impossible to answer this as $\lambda$ can be used to signify a variety of things.

What is SMO mean in SVM?

Sequential Minimal Optimization. It is one of the common methods to efficiently solve the optimization problem associated to training SVMs.

$\endgroup$
  • $\begingroup$ The lamda I meant is in the frst line of equation. something similar with lamda..And for rbf kernel, there are many literature said that this kernel does not separate the class linearly?? So does it mean, that kernel is not always separate data linearly in SVM? $\endgroup$ – user3507478 Apr 9 '14 at 18:17
  • $\begingroup$ in the first line of equation is not clear at all. SVM always separates linearly in feature space. By using a kernel, such as RBF, the feature space is induced via a nonlinear transformation and as such the linear separation in feature space is nonlinear in input space. $\endgroup$ – Marc Claesen Apr 9 '14 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.