Which algorithm is implemented in sklearn's SVM method?

Question

I'd like to know which exact version of svm is implemented in slearn. The references section on sklearn's svm page cites libsvm package and a paper from 1999 which is about comparing classification error rate and likelihood scores. The libsvm paper talks about the original svm paper by Vapnik. But I couldn't figure out which exact formulation of svm has been implemented.

The libsvm paper cites a paper titled Fast Training of Support Vector Machines Using Sequential Minimal Optimization that solves svm using a method called sequential minimal optimization. Is this the algorithm that sklearn actually calls under the hood or is it a different version of this method?

Answer 1

Yes - the implementation there is based on libsvm - which does indeed implement Platt's SMO - you can see the details in this paper.

Answer 2

As you noticed, the documentation says it uses LibSVM, but if in doubt, check the source code:

class SVC(BaseSVC):
    """C-Support Vector Classification.
    The implementation is based on libsvm. The fit time scales at least
    [...]

which inherits from

class BaseSVC(ClassifierMixin, BaseLibSVM, metaclass=ABCMeta):
    """ABC for LibSVM-based classifiers."""

    [...]

that inherits from

class BaseLibSVM(BaseEstimator, metaclass=ABCMeta):
    """Base class for estimators that use libsvm as backing library.
    This implements support vector machine classification and regression.
    [...]

and here we can find the fit method

    def fit(self, X, y, sample_weight=None):
        [...]
        fit = self._sparse_fit if self._sparse else self._dense_fit
        if self.verbose:
            print("[LibSVM]", end="")

        seed = rnd.randint(np.iinfo("i").max)
        fit(X, y, sample_weight, solver_type, kernel, random_seed=seed)
        [...]

which uses

    def _dense_fit(self, X, y, sample_weight, solver_type, kernel, random_seed):
        [...]
            self.fit_status_,
            self._num_iter,
        ) = libsvm.fit(
            X,
            y,
        [...]

or

    def _sparse_fit(self, X, y, sample_weight, solver_type, kernel, random_seed):
        [...]
            self.fit_status_,
            self._num_iter,
        ) = libsvm_sparse.libsvm_sparse_train(
            X.shape[1],
            X.data,
        [...]