2
$\begingroup$

I built an SVM in Python that uses the SMO algorithm described in the paper written by John Platt.

If the alphas (Lagrange multipliers) are initialized to zero at the beginning everything works as expected. However, if they are initialized randomly using some probability distribution yielding values between 0 and C (or literally any other distribution), it doesn't work as it should. In fact the actual weight vector at the end of the training phase has huge values which implies that the margin is very small, and on top of that, it can't separate a linearly separable dataset. It looks completely broken.

My question is: Why does it work only when the alphas have initial values of zero? Is there something wrong with my implementation or is it just a consequence of the way the SMO algorithm works (the paper from John Platt doesn't mention anything about the initial values of the alphas)?

$\endgroup$

1 Answer 1

4
$\begingroup$

The SMO implementation assumes that the constraint $\sum_{i=1}^n \alpha_i y_i = 0$ is maintained throughout the optimization process. The points are selected in pairs and the $\alpha_i$ values modified so as to maintain that constraint (i.e., if the points are selected from the same class then one value is increased and the other decreased, otherwise both must be updated in the same direction). If you are creating them randomly, you are violating the constraint from the beginning and this constraint will never be satisfied.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.