Question about the formulation of SVM

Question

The formulation of the SVM optimization problem is:

\begin{equation} \begin{aligned} & max_{w,b} \frac{1}{||w||} \\ & \text{ subject to } \\ & y_i(w^{T}x_i+b) \geq 1 \end{aligned} \end{equation}

What I do not understand is why do we use $w^Tx_i+b=1$ in the setup. My question is specifically about why 1? I understand that $w^Tx_i+b$ is the equation of a hyperplane and multiplying it by binary class labels $y_i \in \{-1,1\}$ we get the inequality but why do we initially not use $w^Tx_i+b = 2$ or 0 or any number. I am assuming we can adjust for this number since we have b as a hyperparameter.

Thank you!

Answer 1

The goal of a hard-margin SVM (you didn't mention slack variables, $\xi$, for soft-margin SVMs) is to minimize the Euclidean norm $||\mathbf{w}||$ to impose the inequality $y_i D(\mathbf{x}_i)/||\mathbf{w}||>1$ (Boser et al, 1992). This is accomplished by using both the margin value $M$ and the weight vector $\mathbf{w}$ and enforcing the constraint $M ||\mathbf{w}||=1$, which when solving for the optimal margin gives $M=1/||\mathbf{w}||$.

For hard-margin SVMs, the unconstrained Lagrangian function is also \begin{equation} L({\bf w},b,\boldsymbol{\alpha})=\frac{1}{2}{\bf w}^T{\bf w} - \sum_{i=1}^n \alpha_i [y_i ({\bf w}^T{\bf x}_i + b) - 1 ]\\ \end{equation}

$ s.t.\quad \quad \alpha_i \geq 0, \quad y_i ({\bf w}^T{\bf x}_i + b) - 1=0$

Reference:

B.E. Boser, I.M. Guyon, V.N. Vapnik. A training algorithm for optimal margin classifiers. $\textit{Proc. 5th Annual Work. Comp. Learning Theory (COLT'95),}$ pp. 144-152. New York (NY), ACM Press, 1992.