5
$\begingroup$

Just curious about two options in the scikits SVM class.

Does anyone know what scale_C and shrinking do? Not much in the documentation unfortunately.

$\endgroup$
9
$\begingroup$

I realize this is a super old question, but I ran into this same thing today, and found this document. Section 7.3, which describes shrinkage as implemented in libSVM (around which sklearn's SVM is a wrapper), begins with the following useful blurb:

The shrinking technique reduces the size of the problem by temporarily eliminating variables α_i that are unlikely to be selected in the SMO working set because they have reached their lower or upper bound (Joachims, 1999). The SMO iterations then continues on the remaining variables. Shrinking reduces the number of kernel values needed to update the gradient vector (see algorithm 6.2, line 8). The hit rate of the kernel cache is therefore improved.

So basically libSVM optimizes over a subset of the Lagrange multipliers α_i. As many of these are typically zero in a given problem, this is often a safe heuristic to adopt. Much more information can be found in the linked document.

$\endgroup$
5
$\begingroup$

scale_C=True means that the C parameter of the SVM problem is scaled with the number of samples. This is the default in libSVM and liblinear, however if you train models with a widely-varying number of samples it means that a single value of C will not be adequate for all the models. For this reason, we advocate using scale_C=False. It is documented in the list of parameters of the SVM objects.

I must confess that I do not know what the shrinking heuristic is, and it is not properly documented in the libSVM documentation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.