I am a little confused when trying to find the gradient for the multiclass hinge loss:
$$l(y) = \max( 0, 1 + \underset{r \neq y_i}{ \max } W_r \cdot x_i - W_{y_i} \cdot x_i)$$
Where $W^{k \times n}$ is the matrix holding in each row the corresponding classifier of each class.
Unless my math is wrong, the gradient of the function is: \begin{equation} \frac{\partial l}{\partial w} = \begin{cases} 0, & W_{y_i}\cdot x_i > 1 + W_r \cdot x_i \\ 0 + 0 - x_i, & \text{otherwise} \end{cases} \end{equation}
Is this ok?
So if I would like to find the $W$ which minimizes the function using the stochastic gradient descend I would need to do: \begin{equation} W_y^{(t+1)} = W_y^{(t)} - \eta x_i \end{equation}
with $\eta$ the learning rate.
Is this a valid procedure to optimize the $l(y)$ function?