According to here, Now knowing the $a_i$ we can find the weights $w$ for the maximal margin separating hyperplane: \begin{align*} w = \sum_{i=1}^{l} a_i y_i x_i \end{align*}

I cannot understand what this says. I have trouble in how to choose $a_i$. I think we must conduct Newton-method, Pegasos, SMO, or stuff like that.

In addition, I want to implement hard-margin (linear-separable) SVM in Python. So I am seeking for the most simplest(easiest) optimization method.

I am very grateful to you if you make some answers taking this into account.


You're correct that finding the $a_i$ requires some form of iterative optimization. The thing that doesn't require optimization is converting from the dual variables $a_i$ to the primal variable $w$, which just follows the formula that you wrote.

| cite | improve this answer | |
  • $\begingroup$ It does make sence.Thank you. And as I wrote, I want the most simplest optimization method, then I was wondering if you could give me some ideas? $\endgroup$ – Jourd Aug 23 '18 at 0:18
  • $\begingroup$ If you actually want to use it for something, use an existing library, e.g. scikit-learn's bindings to libsvm. If you want to implement it yourself for learning purposes, stochastic gradient descent is probably simplest. $\endgroup$ – Dougal Aug 23 '18 at 0:58
  • $\begingroup$ SGD is for unconstrain ploblem, isn't it? $\endgroup$ – Jourd Aug 23 '18 at 22:26
  • $\begingroup$ Ah, yeah, you'd have to implement some kind of workaround for the constraints (not sure if you can easily project?). Or you could implement SMO, I don't know how complicated it is (never done it). $\endgroup$ – Dougal Aug 24 '18 at 2:42
  • $\begingroup$ OK. It is difficult for me to implement SMO. But, Thank you. $\endgroup$ – Jourd Aug 24 '18 at 3:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.