Differentiation in support vector machine 
My problem is we want to minimize $1/2 \cdot \text{Norm}(w)^2$ under suitable constraints $y(i){[w^Tx_i+b]-1}$.
We write the Lagrangian and get the above equations. 
I have a problem in differentiating with respect to $w$ can somebody elaborate on that? How do we get first equation?
 A: $\frac{\partial L}{\partial w} = 0$
is equivalent to:
$\frac{\partial(\frac{1}{2} w^{T}w - \sum_{i=1}^{N}\alpha_{i}(y_{i}(w^{T}\phi(x_{i}) - b) - 1))}{\partial w} = 0$
which comes down to differentiation of four parts:
$\frac{\partial \frac{1}{2} w^{T}w}{\partial w} = \frac{1}{2}*2*w = w$ (quadratic function of $w$)
$\frac{\partial (\sum_{i=1}^{N}a_{i}y_{i}w^{T}\phi(x_{i}))}{\partial w} = \frac{\partial (a_{1}y_{1}w^{T}\phi(x_{1}) + a_{2}y_{2}w^{T}\phi(x_{2}) + \ldots + a_{N}y_{N}w^{T}\phi(x_{N}))}{\partial w} = a_{i}y_{i}\phi(x_{i}) = \frac{\partial a_{1}y_{1}w^{T}\phi(x_{1})}{\partial w} + \frac{\partial a_{2}y_{2}w^{T}\phi(x_{2})}{\partial w} + \ldots + \frac{\partial a_{N}y_{N}w^{T}\phi(x_{N}))}{\partial w} = a_{1}y_{1}\phi(x_{1} + a_{2}y_{2}\phi(x_{2}) + \ldots + a_{N}y_{N}\phi(x_{N})) = \sum_{i=1}^{N}a_{i}y_{i}\phi(x_{i})$ (linear functions of $w$, $\phi$ also doesn't depend on $w$)
$\frac{\partial a_{i}y_{i}b}{\partial w} = [0, \ldots, 0] $ (constants, not depending on $w$)
and
$\frac{\partial (-a_{i})}{\partial w} = [0, \ldots, 0]$ (constant, not depending on $w$)
Which boils down to the result you've presented:
$\frac{\partial(\frac{1}{2} w^{T}w - \sum_{i=1}^{N}\alpha_{i}(y_{i}(w^{T}\phi(x_{i}) - b) - 1))}{\partial w} = [0, \ldots, 0]$
$w - \sum_{i=1}^{N}\alpha_{i}y_{i}\phi(x_{i}) = [0, \ldots, 0]$
$w = \sum_{i=1}^{N}\alpha_{i}y_{i}\phi(x_{i})$
To explain differentiation over a vector take a look at this link. The first derivative can be thus rewritten more precisely as follows:
$\frac{\partial \frac{1}{2}w^{T}w}{\partial w} = [\frac{\partial \frac{1}{2}w^{T}w}{\partial w_{1}}, \frac{\partial \frac{1}{2}w^{T}w}{\partial w_{2}}, \ldots, \frac{\partial \frac{1}{2}w^{T}w}{\partial w_{m}}] = [\frac{\partial \frac{1}{2}[w_{1}, w_{2}, \ldots, w_{m}]^{T}[w_{1}, w_{2}, \ldots, w_{m}]}{\partial w_{1}}, \frac{\partial \frac{1}{2}[w_{1}, w_{2}, \ldots, w_{m}]^{T}[w_{1}, w_{2}, \ldots, w_{m}]}{\partial w_{2}}, \ldots, \frac{\partial \frac{1}{2}[w_{1}, w_{2}, \ldots, w_{m}]^{T}[w_{1}, w_{2}, \ldots, w_{m}]}{\partial w_{m}}] = [\frac{\partial \frac{1}{2}(w_{1}^{2} + w_{2}^{2} + \ldots + w_{m}^{2})}{\partial w_{1}}, \frac{\partial \frac{1}{2}(w_{1}^{2} + w_{2}^{2} + \ldots + w_{m}^{2})}{\partial w_{2}}, \ldots, \frac{\partial \frac{1}{2}(w_{1}^{2} + w_{2}^{2} + \ldots + w_{m}^{2})}{\partial w_{m}}] = [2*\frac{1}{2}w_{1}, 2*\frac{1}{2}w_{2}, \ldots, 2*\frac{1}{2}w_{m}] = 2*\frac{1}{2}*[w_{1}, w_{2}, \ldots, w_{m}] = 2 * \frac{1}{2} * w = w$
Similarly in other cases, we always differentiate a scalar function over a vector $w$ here, which results in a vector of derivatives over each element $w_{j}, 1 \leq j \leq m$.
A: There is a neat trick because:
$\dfrac{\partial}{\partial w}\dfrac{1}{2}\|w\|^2$
$=\dfrac{\partial}{\partial w}\dfrac{1}{2}<w,w>$
We can relabel $w$ as $w_1$ and $w_2$,
$=\dfrac{1}{2}( \dfrac{\partial}{\partial w_1}<w_1,w_2>+\dfrac{\partial}{\partial w_2}<w_1,w_2>)$
$=\dfrac{1}{2}(w_2+w_1)$
$=\dfrac{1}{2}(w+w)$
$=w$
This is useful in the case where we want the derivative of a more complicated expression because it allows us to isolate the variable we are taking the derivative with respect to.  For example, to minimize
$\|XW-Y\|^2$
One of the terms we need to take the derivative of is
$<XW,XW>$
We can use the fact that $tr(AB)=<A^T,B>$ to say
$<XW,XW>=tr((XW)^TXW)$
$=tr(W^TX^TXW)$
$=<X^TXW,W>$
where the derivative in terms of the lone $W$ is now just $X^TXW$! 
