2
$\begingroup$

Can someone help me with the step by step demonstration of the following equivalence used in SVM:

$$maximize: m = \frac{1} {\|w\|} \equiv minimize: m =\frac{1} {2}\|w\|^2 $$

I would be most grateful if related theorems are included or mentioned.

$\endgroup$
1
$\begingroup$

The argument goes like this.

1) the value that maximizes $1/||w||$ is the value that minimizes $||w||$

2) the value that minimizes $||w||$ also minimizes $||w||^2$,

3) which also minimizes $1/2 ||w||^2$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ As far as theorems are concerned, you can justify each of these statements by checking derivatives (Fermat's theorem) and using the chain rule $\endgroup$ – combo Feb 11 at 23:26
1
$\begingroup$

I presume that you are referring to the maximising/minimising values over some range $w \in \mathscr{W}$. The reason this equivalence holds is that you are using a strictly decreasing transformation. You therefore have:

$$\begin{aligned} \underset{w \in \mathscr{W}}{\text{arg max }} \frac{1}{||w||} &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): \frac{1}{||w||} \geqslant \frac{1}{||w'||} \Big\} \\[6pt] &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): ||w'|| \geqslant ||w|| \Big\} \\[6pt] &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): \frac{1}{2} ||w'||^2 \geqslant \frac{1}{2} ||w||^2 \Big\} \\[6pt] &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): \frac{1}{2} ||w||^2 \leqslant \frac{1}{2} ||wl||^2 \Big\} \\[6pt] &= \underset{w \in \mathscr{W}}{\text{arg min }} \frac{1}{2} ||w||^2. \\[6pt] \end{aligned}$$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.