I ran into a problem, when studying on linear classification. my prof. says:
in a linear classification $y=w_0+w_1x_1+w_2x_2$ that depicted on following figure, distance of origin to decision boundary is equal to $ \frac {|w_0|}{|w|}$.
any expert can tell me, how this will be calculated?
thanks to all.
The decision boundary by definition has $y=0$, so all of its points satisfy $-w_0 = w_1 x_1 + w_2 x_2$ $(\star)$. The distance from the origin to a point $x = (x_1, x_2)$ is $\lVert x \rVert$; the distance from the origin to the decision boundary is thus the minimum of $\lVert x \rVert$ among points satisfying $(\star)$.
Note that the right hand side of $(\star)$ is $w^T x = \lVert w \rVert \lVert x \rVert \cos \theta$, where $\theta$ is the angle between $w$ and $x$. Since this value is constant at $-w_0$, $\lVert x \rVert$ is minimized when $\lvert \cos\theta \rvert$ is maximized, i.e. 1, when $w$ and $x$ are parallel or antiparallel depending on the sign of $w_0$. You can also see this from the picture; there, $w_0 < 0$ so $w$ and $x$ are parallel.
Thus, at that point, we have $\lVert x \rVert = \frac{- w_0 }{\lVert w \rVert \cos \theta}$, with $\cos \theta \in \{-1, 1\}$. Taking absolute values gives $\lVert x \rVert = \frac{\lvert w_0 \rvert}{ \lVert w \rVert }$, the distance from the origin to the decision boundary as claimed.
