Skip to main content
3 of 3
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/

The Lagrange multipliers in the context of SVMs are typically denoted $\alpha_i$. The fact that one often observes that most $\alpha_i=0$ is a direct consequence of the Karush-Kuhn-Tucker (KKT) dual complementarity conditions:

enter image description here

Since $y_i(\mathbf{w}^T\mathbf{x}_i+b) = 1$ iff $\mathbf{x}_i$ is on the SVM decision boundary, i.e. is a support vector assuming $\mathbf{x}_i$ is in the training set, and in most cases few training vectors are support vectors, as whuber pointed out in the comments, it means that most $\alpha_i$ are 0 or $C$.


Andrew Ng's CS229 Lecture notes on SVMs introduces the Karush-Kuhn-Tucker (KKT) dual complementarity conditions:

enter image description here

enter image description here

enter image description here

enter image description here

Note that we can create some case where all vectors in the training set are support vectors: e.g. see this Support Vector Machine Question.

Franck Dernoncourt
  • 47.6k
  • 33
  • 179
  • 293