From what I've learned, training a (hard-margin) linear SVM on training data gives weights $w$ and intercept $b$ such that it forms hyperplane $\{x: x^Tw + b = 0\}$- this is the hyperplane that maximizes the margin between linearly separable data. Additionally, $w$ and $b$ are formed such that $\{x: x^Tw + b = -1\}$ and $\{x: x^Tw + b = 1\}$ give the two margin hyperplanes that each intersect with at least 1 support vector point.

However, I've read from many sources that the weights vector $w$ is supposed to be a unit vector ($||w|| = 1$). I'm confused on why/when this is necessary. Isn't the margin width specifically calculated by $\frac{2}{||w||}$? The width of the margin can't always be 2?

For example, take these set of points below, which each have two features $x_1, x_2$:

enter image description here

The bold line is the max-margin hyperplane represented by $w = [\frac{1}{4}, -\frac{1}{4}], \alpha=-\frac{3}{4}$.

Of course, $w$ here is normal to the hyperplane, but is not a unit vector. What information would normalizing $w$ give us?

  • $\begingroup$ Can you link to some of these "many sources" which claim that $w$ needs to be a unit vector? $\endgroup$ – Igor F. Mar 31 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.