# Compute gradient for the SVM

I am working on Java implementation of the SVM.

Suppose that I have the following loss function that needs to be minimized on weight-space:

$$\frac{1}{N} \sum\limits_{i=1}^N \left(\frac{\lambda}{2}\cdot \|w\| + \max(0,1-y_i \cdot f(x_i) )\right)$$

where

• $N$ - number of the features
• $x_i$ - the particular feature (basically it is a vector with the length $M$);
• $w$ - weights that have to be found by gradient descent method;

Based on the following presentation http://www.robots.ox.ac.uk/~az/lectures/ml/lect2.pdf

The update rules are the following:

• $w_{t+1}\leftarrow w_t-\eta (\lambda w_t-y_i x_i)$, when $y_i (w^Tx_i+b)<0$
• $w_{t+1}\leftarrow w_t-\eta \lambda w_t$, otherwise

However, I don't understand the following:

$y_i$ might contain only ${0,1}$ - it is scalar, but $x_i$ is vector. How can I simply multiply vector by a scalar and get a vector?

Is it simply the following multiplication

$y_i x_i = (y_i x_i^1,..., y_i x_i^M)$?

• I think y_i should be either 1 or -1 under the SVM context. – Sean Do Apr 24 '18 at 9:13

## 1 Answer

Yes, it is as you conjectured, i.e., each element of the vector is multiplied by the scalar, with the understanding that if $x_i$ is a column vector, as it is in this case, then $y_i x_i$ is also a column vector (which is not clear from your notation).

Given that $y_i$ is either 0 or 1, then $y_i x_i$ will be the M by 1 column vector of zeros if $y_i = 0$, and will be the M by 1 column vector $x_i$, if $y_i = 1$.