In Hard margin support vector machine we have equation of Hyper-plane
$W^T*X+b = 0 $, I want to know why for a point $X_n$ not on Hyper-plane
$W^T*X+b = 0$ we take
$|W^T*X_n+b| = 1$
why do we normalize to 1
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
If I understand your question correctly, you wonder why the boundary hyperplanes of the separating region, defined as $\{x: w^Tx+b=\pm k\}$, have a fixed value for $k$ (namely 1 in most texts).
Short answer: If $k$ is treated as a parameter to be estimated, then the problem becomes underdetermined.
Exhibit: If $k$ would be treated as an unknown parameter, any candidate solution $(w,b,k)$ would imply a set of candidate solutions $\{(\lambda w, \lambda b, \lambda k) : \lambda\in\mathbb R\}$ which all amount to the same practical solution (that is, the same margin region).
Now, given that the loss function for the hard-margin problem is just $f(w)=\|w\|^2\geq 0$, letting $\lambda\downarrow 0$ shows that any separating region would be optimal.
