What is meant by "number of support vectors" in the SVM implementation of scikit-learn I noticed that decreasing the C regularization parameter tends to increase n_support_ in the solution provided by sklearn.svm.SVC.
What does it mean for "number of support vectors" to be greater than 2 in a linearly separable, binary classification problem solved by a SVC? I'm coming from the understanding that support vectors are the closest ones to the discriminating hyperplane. Unless there are ties, I can only see there being one for each class.
 A: You are correct that the “number of support vectors” are the training points directly used to find your linear classification boundary. By decreasing the C variable, you are decreasing the amount of variance allowed in your classification boundary, as more support vector are used.
By having more than 2 support vectors in many cases, the variance of that boundary can substantially decrease.
This C variable is the cost of adding support vectors, and can be thought of as adjusting the importance of your training data to the boundary; so if you increase C you are making the boundary less reliant on the training data (and more reliant on the points closest between classes) whereas decreasing C makes the boundary more reliant on observations from the training data.
A higher C makes the boundary more sensitive to uniqueness in the training data. So if you happen to use a sample that is not representative of the population, a high C value SVC will be susceptible to a higher miss-classification rate.
