0
$\begingroup$

I try to train a linear SVM using package e1071, with cost=10 on the following data:

x <- matrix(c(-0.97, -0.69,-1.14,-0.93,0.26,1.14,0.76,1.32,- 0.79,1.04,-.80,-.97,-1.09,-1.09,0.63,1.09,0.92,1.49,-0.52,0.34),10,2)
colnames(x) <- c("X1","X2")
y <- c(-1,-1,-1,-1,1,1,1,1,1,-1)
svm2 <- svm(x,y,type="C-classification", kernel="linear", cost=10, scale=FALSE)

The data looks like: enter image description here

In theory, point E(0.26,0.63) should also be a support vector. But svm() does not return it as a support vector, but considers point T(1.14,1.09) as SV. This happens also when I change the kernel to "radial".

In theory, E should be a support vector, while point T(1.14, 1.09) not, if I understand correctly the theory.

Might this inconsistency be due to the fact that I have only 10 observations?

$\endgroup$
2
  • $\begingroup$ How this is bad? You have one misclassified case. $\endgroup$
    – Tim
    Aug 29 '17 at 7:54
  • $\begingroup$ I do not understand why E is not a support vector, given that it lies between the margins. $\endgroup$
    – ralucaGui
    Aug 29 '17 at 8:11
0
$\begingroup$

The main issue is that there exists a linear seperation of both classes without any loss. The correct support vectors should be S, T, A, D with a linear kernel. I do not see any inconsistency with E not beeing a support vector.

The plot illustrates the proper linear seperation of both classes (roughly, not done with svm), support vectors marked in red.

enter image description here

$\endgroup$
2
  • $\begingroup$ Thank you. Can you please let me know what software and which package did you use to get your results? $\endgroup$
    – ralucaGui
    Aug 29 '17 at 8:09
  • $\begingroup$ I used Python (&matplotlib) to plot your data and I used my basic knowledge of SVM to identify the support vectors. Anyhow I would recommend to use Python and the Scikit-Learn package (well documented with a lot of further information). Although I do not use R, yet I never heard anything bad about it - so it seems to be just as fine. $\endgroup$ Aug 29 '17 at 8:12
0
$\begingroup$

It sounds like you're trying to apply the intuitions of a hard-margin SVM to the results of a soft-margin SVM and being led astray by the differences.

I can tell you're using a soft-margin SVM because you provided a "cost=10" parameter; the e1071 R library does not appear to support a hard-margin classifier directly, but you can approximate it by using a suitably huge cost parameter, say 1e10.

When the cost is low, a soft-margin SVM will sometimes choose to misclassify one or more points if it means it can get a wider margin. Perhaps surprisingly, they may do this even when the data are truly linear separable and a "perfect" solution could be found. If you increase the cost to 100 or 1,000 it will behave more like the hard-margin version and you will see it correctly classify all training examples.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.