Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I was watching the MIT open course on machine learning. In the session on SVM, the professor derived that the margin is $\frac{1}{\|w\|}$. However, the professor then said for the convenience of mathematics, to maximize $\frac{1}{\|w\|}$, which implies to minimize $\|w\|$, is minimizing $\frac{\|w\|^2}{2}$.
What is the rationale behind this substitution?
Notice that $\frac{1}{x}$ is a decreasing function over the positive domain and $\frac{x^2}{2}$ is an increasing function over the non-negative domain.
If $g$ is a decreasing function (a function where as the input increases, the output decreases). Maximizing $f_1(x)$ is equivalent to minimizing $g(f_1(x))$. Here $f_1(w)=\frac{1}{\|w\|}$, and $g(x)=\frac{1}{x}$, hence $g(f_1(w))=\|w\|$.
If $h$ is an increasing function (a function where if the input increases, the output increases, similarly, if the input decreases, the output decreases). Minimizing $f_2(x)$ is equivalent to minimizing $h(f_2(x))$. Here $f_2(w)=\|w\|$ and $h(x)=\frac{x^2}{2}$, hence $h(f_2(w))=\frac{\|w\|^2}{2}$.
