# Can anyone help me with understanding ||w||

I am studying Machine Learning (SVMs), and I found this $||w||$ in front of me. I have not seen it again, and Google does not allow me to search for it. Does anyone knows what is $w$, what is $||w||$, how it is called, and why we need that?

Many thanks

• ...the norm of w. – Elvis Nov 24 '13 at 13:17
• Hi John, welcome to cross validated. When asking questions, please take care that they are actually answerable: so if you ask about a specific symbol, it is probably best if you include some context. You can cite the relevant part the paper/book where you found it, and give a link to the full source. After all, everyone is free to define his own $w$... – cbeleites supports Monica Nov 24 '13 at 13:53
• @JohnSmith: well, then put the equations the prof had where the $w$ appeared (I'd expect it to be the normal vector of the separating hyperplane, as Marc Claesen says). And maybe tell us that this was in a lecture. – cbeleites supports Monica Nov 24 '13 at 14:04
• @cbeleites He had a graph with points and a line separating them, and he wrote y(x) = w^Tx + w0. And at the graph he also had a line vertical with the hyperplane starting from the start of axes, which he said that it was the vector w, and in addition a point x with an y(x)/||w|| next to it. To be honest, I did not understand much of these things... And there is not any other explanation at the slides.. – Jim Blum Nov 24 '13 at 14:10
• For similar questions, try e.g. en.wikipedia.org/wiki/List_of_mathematical_symbols – Nick Cox Nov 24 '13 at 14:22

$\Vert w \Vert$ represents the norm of the vector $w$.

There are different norms that you can consider. When it is not said, it's often the euclidian norm, the norm 2:

$$\Vert w \Vert = \Vert w \Vert_2 = \sqrt{w_1^2 + \cdots + w_n^2}$$

Here are some other norms:

• norm 1: $$\Vert w \Vert_1 = |w_1| + \cdots + |w_n|$$
• norm $p$: $$\Vert w \Vert_p = \left( |w_1|^p + \cdots + |w_n|^p \right)^{1/p}$$
• infinite norm: $$\Vert w \Vert_\infty = \lim_{p \to \infty} \Vert w \Vert_p = max\left\{ |w_1| + \cdots + |w_n| \right\}$$

Right, one way to think of $w$ is as a vector representing a separating hyperplane. However, in order to understand its usefulness, you better consider it a regularization value: $||w||$ is a measure for the complexity of the model represented by the SVM. The idea is to find a model that on one hand accurately models the training data, and on the other hand has a minimal complexity, because simple models tend to generalize better. So the optimization term of the SVM has two components, with $||w||$ representing the complexity.