Define the hard-margin optimization problem for the seperable case as:

\begin{align} \max_{\mathbf{w},b,M} \quad & M \\ \textrm{s.t.} \quad & \frac{1}{\lvert \lvert \mathbf{w} \rvert \rvert} y_i(\mathbf{w}\cdot\mathbf{x_i} + b) \geq M \geq 0 \quad \forall{\mathbf{x_i}}, i=\{1,\dots,m\} \\ \end{align}

where $M=\min_{i=1,\dots,m}M_i = \min_{i=1,\dots,m}y_i\frac{\mathbf{w}\cdot\mathbf{x_i}+b}{\lvert\lvert \mathbf{w}\rvert\rvert}$

The objective function $y_i\frac{\mathbf{w}\cdot\mathbf{x_i}+b}{\lvert\lvert \mathbf{w}\rvert\rvert}$ is invariant under scaling if we scale b and w in the same way, which means that we can choose $\lvert \lvert \mathbf{w} \rvert \rvert = \frac{1}{M}$.

Here comes my question:

When I change the scaling for w, I must also change the change b accordingly. This means that I must divide b by M as well which would lead to the following equation:

\begin{align} \max_{\mathbf{w},b} \quad & \frac{1}{\lvert \lvert \mathbf{w} \rvert \rvert} \\ \textrm{s.t.} \quad & y_i(\frac{1}{M}\frac{1}{\lvert \lvert \mathbf{w} \rvert \rvert}\mathbf{w}\cdot\mathbf{x_i} + \frac{b}{M}) \geq 1 \quad \forall{\mathbf{x_i}}, i=\{1,\dots,m\} \end{align}

However, all derivations I have seen have the following

\begin{align} \max_{\mathbf{w},b} \quad & \frac{1}{\lvert \lvert \mathbf{w} \rvert \rvert} \\ \textrm{s.t.} \quad & y_i(\mathbf{w}\cdot\mathbf{x_i} + b) \geq 1 \quad \forall{\mathbf{x_i}}, i=\{1,\dots,m\} \end{align}

What am I doing wrong?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.