Just curious about two options in the scikits SVM class.
Does anyone know what
shrinking do? Not much in the documentation unfortunately.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
I realize this is a super old question, but I ran into this same thing today, and found this document. Section 7.3, which describes shrinkage as implemented in libSVM (around which sklearn's SVM is a wrapper), begins with the following useful blurb:
The shrinking technique reduces the size of the problem by temporarily eliminating variables α_i that are unlikely to be selected in the SMO working set because they have reached their lower or upper bound (Joachims, 1999). The SMO iterations then continues on the remaining variables. Shrinking reduces the number of kernel values needed to update the gradient vector (see algorithm 6.2, line 8). The hit rate of the kernel cache is therefore improved.
So basically libSVM optimizes over a subset of the Lagrange multipliers α_i. As many of these are typically zero in a given problem, this is often a safe heuristic to adopt. Much more information can be found in the linked document.
scale_C=True means that the C parameter of the SVM problem is scaled with the number of samples. This is the default in libSVM and liblinear, however if you train models with a widely-varying number of samples it means that a single value of C will not be adequate for all the models. For this reason, we advocate using
scale_C=False. It is documented in the list of parameters of the SVM objects.
I must confess that I do not know what the shrinking heuristic is, and it is not properly documented in the libSVM documentation.