Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I was studying Support Vector Machines and I've got stuck with this relation regarding the weight vector of the hyperplane.
$w=\sum\limits_{i\in I}^{} y_i x_i$
For reference, I'm studying from the Alex Smola slides and this relation is not justified.
That particular slide is about Perceptron algorithm, where initially $w$ is $0$ and you update it for each misclassified sample with the following update rule (there are slightly changed versions of this, but sticking with the slides): $$w\leftarrow w+y_ix_i$$ Because we start from $0$ and every update made is in terms of $y_ix_i$, the final version of the weights will be a linear combination of the input samples, i.e. something like $$w=\sum m_i y_i x_i$$
where $m_i$ is the number of times in the learning loop we updated for sample $i$. The concept of linear combination is correct, but the equation provided is wrong. If it were, we could have solved for $w$ directly, without the need of iterating until convergence: $$w=\sum y_i x_i=\sum_{y_i=1} x_i-\sum_{y_i=-1} x_i$$
