I have been thinking of this case as i was reading other questions and answers about SVM.

A question was raised about an SVM model having 1000 data points and 800 support vectors. The OP used a linear kernel. However, this is something that confuses me. If an SVM is supposed to maximise the margin between the 2 classes. How would the increase in number of support vectors feature in this calculations?

How to ensure that all the vectors are maximimum margin from each other?


I would suggest you to look at SVM from another point of view: It is minimizing hinge loss with L2 regularization. This interpretation works better than maximize the margin interpretation when data is not linearly separable.

Increasing the regularization will increase number of the support vectors.

svp1 <- ksvm(d$x,d$classes,type="C-svc",kernel="polydot",C=0.1)
svp2 <- ksvm(d$x,d$classes,type="C-svc",kernel="polydot",C=10000)

enter image description here

In above simulation we are changing the regularization parameter $C$ to have different numbers of support vectors (shown in black in the plot) in a toy data set.

  • $\begingroup$ minimizing the hinge loss would require as many data points as needed? Can i have a more detailed explaination $\endgroup$ – aceminer Oct 4 '16 at 14:13
  • $\begingroup$ @aceminer would the example help? libsvm paper section 2.1 will give you math details. $\endgroup$ – hxd1011 Oct 4 '16 at 14:22
  • $\begingroup$ So just to confirm is this in high dimensional data? Or data in 2d. Not too good with r $\endgroup$ – aceminer Oct 4 '16 at 14:42
  • $\begingroup$ @aceminer 2D data. 2 classes, ground truth label are circle and triangle. $\endgroup$ – hxd1011 Oct 4 '16 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.