0
$\begingroup$

I'm trying to understand SVM (Support Vector Machines) and there is a single technicality I don't understand.

In Wikipedia (and several other literature), the margins are described by the equations

$$\vec { w } \cdot \vec { x } - b = 1$$ and $$\vec { w } \cdot \vec { x } - b = -1$$

Wikipedia states these equations holds with a normalized or standardized dataset. What does it mean to be normalized and why must the equations be equal to $-1$ and $1$?

I also don't understand what $\vec { x }$ is in that case? Is $\vec { x }$ the nearest point to the separating hyperplane?

Any help is appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

$\vec{x}$ is here a variable. Each of the two equation holds true for a certain set of points. If it does for $\vec{x}$, then $\vec{x}$ is one of the points on the border.

I think normalization is a transofrmation applied on your data that makes it ressemble a normal distribution, but I am not exactly sure about the details.

Finally, there is nothing special about 1 and -1, we could build models based on other numbers (indeed, we get the exact same separation if we later change it to 1/2 and -1/2 for instance). However, that is often the name we label classes, so it is practical. The point is that they are two separate (different) values, thus making it a "hard" (there is space in-between) border

Improve this answer
$\endgroup$
5
  • $\begingroup$ I don't think the reason they use $-1$ and $1$ in these equations is because of the labels. Because the separation should not change even if the change the labels name? The seperation will change, like you said, if we change the labels? $\endgroup$
    – newToMachineLearning
    May 27, 2019 at 16:26
  • $\begingroup$ Of course it's not, you can pick other numbers! That's why $b$ is in there! If you want the right-hand side of the equations to be $-15$ and $\pi^2)$, change the module (not the direction) of $\vec{w}$ and $b$ accordingly. You will get the same separation. It's just easy to say that "negative labeled" points are on the negative side and "positive labeled" points are on the positive one $\endgroup$
    – David
    May 27, 2019 at 19:44
  • $\begingroup$ Actually, there is something I still don't understand. Why would the module change? (By module, you mean lenght right?) If we change the labels, the coordinates will still be the same? Am I wrong? And therefore the "length" will still remain the same? $\endgroup$
    – newToMachineLearning
    May 28, 2019 at 7:48
  • $\begingroup$ Yes, by "module" I mean "length", and labels won't change coordinates. I was talking about changing the right-hand side of the equations (while careless about the labels, as far as the model is concerned the groups could be called "pretty" and "ugly") I wouldn't really overcomplicate myself that much. Try seeing with an easy example in 2D or 3D that given two parallel lines/planes that separate the 2D/3D space in three chunks (below the lower, above the higher or between the two), you can find a vector $\vec{w}$ such that the value $\vec{w} \cdot \vec{x}$ determines where $\vec{x}$ will land $\endgroup$
    – David
    May 28, 2019 at 8:05
  • $\begingroup$ Then, see what happens when you pick a different $b$ or a multiple of $\vec{w}$ $\endgroup$
    – David
    May 28, 2019 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.