# Regarding redundant training data in building SVM-based classifier

To build a SVM-based classifier, I have a training data set consisting of N data points. Some of them are redundant. For instance, there have 50 data points which are exactly the same, and there have other 100 data points which are exactly the same. I have two choices, remove the redundant ones and construct the reduced data set; keep the original data set. Will the resulting classifier be different after applying these two different choices?

-

## 1 Answer

If you are using hard margins, there is no difference because the best margin is the same either way.

If you are using soft margins, then duplicating a data point can matter since the penalty is a sum over data points within the margin, and duplicating these data points affects the size of the penalty.

Here are $1$-dimensional pictures showing what might be the best soft-margin classifiers without and with duplication.

$XXX~~~~~~~~~~X~|~~~~~~~~~~~OOOO$

$XXX~~~~~~XXX~~~~~|~~~~~OOOO$

-
 thanks. Can you explain why the boundary is moved to a position being more closer to 'oooo' when X has more copies. – bit-question Dec 4 '12 at 15:02 In a soft-margin classifier, you accept some amount of examples times normalized distance within the margin. Duplicating the $X$ in the middle may make the penalty too high, greater than the acceptable threshold, so that you have to decrease the margin by a lot at the old position. This makes the new position a better soft-margin classifier when the $X$s in the middle are duplicated. – Douglas Zare Dec 4 '12 at 15:34 very nice explanations, thanks. – bit-question Dec 4 '12 at 20:20