2
$\begingroup$

I am trying to understand SVM from very basic level. I encountered this statement that large margin tries to improve the test error i.e the generalization ability of the classifier. I read about it further in this but still I don't get the point.

Can someone please explain this to me ?

Improve this question
$\endgroup$

3 Answers 3

Reset to default
1
$\begingroup$

Small margins will allow it do draw very complex decision boundaries that take many sharp turns. Larger margins don't allow that because the margin wouldn't fit between the data points that you are trying to turn around.

The more complex the decision boundaries and the more it just represents where single data points lie in your training data, the less likely it will be representative of other data not in your training set: over-fitting.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the answer David. Does "Memory capacity has been decreased" as mentioned in the link provided means the same as learning complex decision boundary if the margin is large has decreased ? $\endgroup$
    – Ashutosh Mishra
    Sep 22, 2017 at 19:55
  • $\begingroup$ yes, less memory capacity if the margin is large $\endgroup$
    – David Ernst
    Sep 22, 2017 at 20:06
1
$\begingroup$

One answer to this question comes from the SRM (Structural Risk Minimization). In fact it has been proven in (Vapnik, 2000) that the VC dimensions of a separating hyperplane is bounded by the margin.

A more in depth discussion can be found here.

Improve this answer
$\endgroup$
0
$\begingroup$

Because a larger elastic margin produces a more stable result, by adding some bias for less variance error in the training. It helps avoiding overfitting as well.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks Tonca but could you please be more general, in terms of layman language !! $\endgroup$
    – Ashutosh Mishra
    Sep 22, 2017 at 19:50
  • $\begingroup$ In other words, the larger margin is less sensible to noise or outliers. But you should really read about the bias-variance trade-off. It is one the most important concepts of ML. $\endgroup$
    – Tonca
    Sep 22, 2017 at 19:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.