5
$\begingroup$

I ran into a problem, when studying on linear classification. my prof. says:

in a linear classification $y=w_0+w_1x_1+w_2x_2$ that depicted on following figure, distance of origin to decision boundary is equal to $ \frac {|w_0|}{|w|}$.

enter image description here

any expert can tell me, how this will be calculated?

thanks to all.

$\endgroup$

1 Answer 1

1
$\begingroup$

The decision boundary by definition has $y=0$, so all of its points satisfy $-w_0 = w_1 x_1 + w_2 x_2$ $(\star)$. The distance from the origin to a point $x = (x_1, x_2)$ is $\lVert x \rVert$; the distance from the origin to the decision boundary is thus the minimum of $\lVert x \rVert$ among points satisfying $(\star)$.

Note that the right hand side of $(\star)$ is $w^T x = \lVert w \rVert \lVert x \rVert \cos \theta$, where $\theta$ is the angle between $w$ and $x$. Since this value is constant at $-w_0$, $\lVert x \rVert$ is minimized when $\lvert \cos\theta \rvert$ is maximized, i.e. 1, when $w$ and $x$ are parallel or antiparallel depending on the sign of $w_0$. You can also see this from the picture; there, $w_0 < 0$ so $w$ and $x$ are parallel.

Thus, at that point, we have $\lVert x \rVert = \frac{- w_0 }{\lVert w \rVert \cos \theta}$, with $\cos \theta \in \{-1, 1\}$. Taking absolute values gives $\lVert x \rVert = \frac{\lvert w_0 \rvert}{ \lVert w \rVert }$, the distance from the origin to the decision boundary as claimed.

$\endgroup$
4
  • $\begingroup$ thanks, some times I get $|w_0|$ or $ \frac{w_0}{|w|}$ so both of them is false ! thanks. $\endgroup$ Mar 29, 2015 at 8:58
  • $\begingroup$ Well, sometimes people constrain $\lVert w \rVert = 1$ for certain things, in which case the first one would be true. $\endgroup$
    – Danica
    Mar 29, 2015 at 9:01
  • $\begingroup$ in the mentioned problem $||w||=1$ ? am I right? $\endgroup$ Mar 29, 2015 at 9:05
  • $\begingroup$ The picture doesn't make that clear. $\endgroup$
    – Danica
    Mar 29, 2015 at 9:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.