1
$\begingroup$

The formulation of the SVM optimization problem is:

\begin{equation} \begin{aligned} & max_{w,b} \frac{1}{||w||} \\ & \text{ subject to } \\ & y_i(w^{T}x_i+b) \geq 1 \end{aligned} \end{equation}

What I do not understand is why do we use $w^Tx_i+b=1$ in the setup. My question is specifically about why 1? I understand that $w^Tx_i+b$ is the equation of a hyperplane and multiplying it by binary class labels $y_i \in \{-1,1\}$ we get the inequality but why do we initially not use $w^Tx_i+b = 2$ or 0 or any number. I am assuming we can adjust for this number since we have b as a hyperparameter.

Thank you!

$\endgroup$
1
$\begingroup$

The goal of a hard-margin SVM (you didn't mention slack variables, $\xi$, for soft-margin SVMs) is to minimize the Euclidean norm $||\mathbf{w}||$ to impose the inequality $y_i D(\mathbf{x}_i)/||\mathbf{w}||>1$ (Boser et al, 1992). This is accomplished by using both the margin value $M$ and the weight vector $\mathbf{w}$ and enforcing the constraint $M ||\mathbf{w}||=1$, which when solving for the optimal margin gives $M=1/||\mathbf{w}||$.

For hard-margin SVMs, the unconstrained Lagrangian function is also \begin{equation} L({\bf w},b,\boldsymbol{\alpha})=\frac{1}{2}{\bf w}^T{\bf w} - \sum_{i=1}^n \alpha_i [y_i ({\bf w}^T{\bf x}_i + b) - 1 ]\\ \end{equation}

$ s.t.\quad \quad \alpha_i \geq 0, \quad y_i ({\bf w}^T{\bf x}_i + b) - 1=0$

Reference:

B.E. Boser, I.M. Guyon, V.N. Vapnik. A training algorithm for optimal margin classifiers. $\textit{Proc. 5th Annual Work. Comp. Learning Theory (COLT'95),}$ pp. 144-152. New York (NY), ACM Press, 1992.

$\endgroup$
1
  • $\begingroup$ thanks for your answer @nxglogic, I am wondering why the hyperplane equation we use is $w^Tx_i + b= 1$ instead of $w^Tx_i + b = 0$ $\endgroup$ – Kaan Yolsever Apr 21 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.