I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane.

$w=\sum\limits_{i\in I}^{} y_i x_i$

For reference, I'm studying from the Alex Smola slides and this relation is not justified.


That particular slide is about Perceptron algorithm, where initially $w$ is $0$ and you update it for each misclassified sample with the following update rule (there are slightly changed versions of this, but sticking with the slides): $$w\leftarrow w+y_ix_i$$ Because we start from $0$ and every update made is in terms of $y_ix_i$, the final version of the weights will be a linear combination of the input samples, i.e. something like $$w=\sum m_i y_i x_i$$

where $m_i$ is the number of times in the learning loop we updated for sample $i$. The concept of linear combination is correct, but the equation provided is wrong. If it were, we could have solved for $w$ directly, without the need of iterating until convergence: $$w=\sum y_i x_i=\sum_{y_i=1} x_i-\sum_{y_i=-1} x_i$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.