0
$\begingroup$

I am studying Machine Learning (SVMs), and I found this $||w||$ in front of me. I have not seen it again, and Google does not allow me to search for it. Does anyone knows what is $w$, what is $||w||$, how it is called, and why we need that?

Many thanks

|cite|improve this question
$\endgroup$
  • $\begingroup$ ...the norm of w. $\endgroup$ – Elvis Nov 24 '13 at 13:17
  • 2
    $\begingroup$ Hi John, welcome to cross validated. When asking questions, please take care that they are actually answerable: so if you ask about a specific symbol, it is probably best if you include some context. You can cite the relevant part the paper/book where you found it, and give a link to the full source. After all, everyone is free to define his own $w$... $\endgroup$ – cbeleites supports Monica Nov 24 '13 at 13:53
  • 3
    $\begingroup$ @JohnSmith: well, then put the equations the prof had where the $w$ appeared (I'd expect it to be the normal vector of the separating hyperplane, as Marc Claesen says). And maybe tell us that this was in a lecture. $\endgroup$ – cbeleites supports Monica Nov 24 '13 at 14:04
  • 1
    $\begingroup$ @cbeleites He had a graph with points and a line separating them, and he wrote y(x) = w^Tx + w0. And at the graph he also had a line vertical with the hyperplane starting from the start of axes, which he said that it was the vector w, and in addition a point x with an y(x)/||w|| next to it. To be honest, I did not understand much of these things... And there is not any other explanation at the slides.. $\endgroup$ – Jim Blum Nov 24 '13 at 14:10
  • 1
    $\begingroup$ For similar questions, try e.g. en.wikipedia.org/wiki/List_of_mathematical_symbols $\endgroup$ – Nick Cox Nov 24 '13 at 14:22
1
$\begingroup$

$\Vert w \Vert$ represents the norm of the vector $w$.

There are different norms that you can consider. When it is not said, it's often the euclidian norm, the norm 2:

$$\Vert w \Vert = \Vert w \Vert_2 = \sqrt{w_1^2 + \cdots + w_n^2}$$

Here are some other norms:

  • norm 1: $$\Vert w \Vert_1 = |w_1| + \cdots + |w_n|$$
  • norm $p$: $$\Vert w \Vert_p = \left( |w_1|^p + \cdots + |w_n|^p \right)^{1/p}$$
  • infinite norm: $$\Vert w \Vert_\infty = \lim_{p \to \infty} \Vert w \Vert_p = max\left\{ |w_1| + \cdots + |w_n| \right\}$$

Here are more details on the Wikipedia page Norm (Mathematics).

|cite|improve this answer
$\endgroup$
0
$\begingroup$

Right, one way to think of $w$ is as a vector representing a separating hyperplane. However, in order to understand its usefulness, you better consider it a regularization value: $||w||$ is a measure for the complexity of the model represented by the SVM. The idea is to find a model that on one hand accurately models the training data, and on the other hand has a minimal complexity, because simple models tend to generalize better. So the optimization term of the SVM has two components, with $||w||$ representing the complexity.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.