# Optimization equivalence

Can someone help me with the step by step demonstration of the following equivalence used in SVM:

$$maximize: m = \frac{1} {\|w\|} \equiv minimize: m =\frac{1} {2}\|w\|^2$$

I would be most grateful if related theorems are included or mentioned.

The argument goes like this.

1) the value that maximizes $$1/||w||$$ is the value that minimizes $$||w||$$

2) the value that minimizes $$||w||$$ also minimizes $$||w||^2$$,

3) which also minimizes $$1/2 ||w||^2$$

• As far as theorems are concerned, you can justify each of these statements by checking derivatives (Fermat's theorem) and using the chain rule – combo Feb 11 at 23:26

I presume that you are referring to the maximising/minimising values over some range $$w \in \mathscr{W}$$. The reason this equivalence holds is that you are using a strictly decreasing transformation. You therefore have:

\begin{aligned} \underset{w \in \mathscr{W}}{\text{arg max }} \frac{1}{||w||} &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): \frac{1}{||w||} \geqslant \frac{1}{||w'||} \Big\} \\[6pt] &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): ||w'|| \geqslant ||w|| \Big\} \\[6pt] &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): \frac{1}{2} ||w'||^2 \geqslant \frac{1}{2} ||w||^2 \Big\} \\[6pt] &= \Big\{ w \in \mathscr{W} \Big| (\forall w'): \frac{1}{2} ||w||^2 \leqslant \frac{1}{2} ||wl||^2 \Big\} \\[6pt] &= \underset{w \in \mathscr{W}}{\text{arg min }} \frac{1}{2} ||w||^2. \\[6pt] \end{aligned}