It was my understanding that in a separable case, SVMs produce the best separation possible and therefore will always produce the same or a better classification rate compared with say, 1NN, mulitlayer perceptron etc.

I've found that in some of my data (1 method in a set of 8), 1NN produces a very slightly better classification rate than SVM (53.5% compared with 52.5%). Obviously this case is not linearly separable as SVM fails to separate it, yet, it does not feel right that 1NN beats SVM.

So my question really is, when is 1NN, MLP etc better than SVM and why?

  • 2
    $\begingroup$ I don't understand your question. You concede that your dataset is not linearly separable -- then why are you surprised that 1-NN beats linear SVM? $\endgroup$
    – cfh
    Apr 15 '15 at 21:25

I guess you're using a linear kernel SVM, which gives linear decision boundaries. k-NN, on the other hand, gives Voronoi shaped decision boundaries, which are non-linear. If your data is not linearly separable, then there are good chances k-NN may outperform linear SVM.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.