Why dual problem coefficients from svm.SVC contain zeros My question is about the output in sklearn.svm.SVC function in Python. Apologies for a software context question but I believe a good number of those who have learnt SVM will have come across this function.
In the sklearn documentation section 1.4.7.1, it is stated that 'These parameters can be accessed through the attributes dual_coef_ which holds the product $y_i \alpha_i$. The array returned by dual_coef_ is of shape (n_classes-1, n_sv).
As I understand, the support vectors are those data points whose $\alpha_i > 0$, thus they will make an impact on the decision boundary which has the form
\begin{align*}
f(x) = \sum_i \alpha_i K(x_i, x)
\end{align*}
Those non-support vectors have their $\alpha_i = 0$.
When I fit the following model however the output array in dual_coef_ does contain zeros. What is the explanation for that? Thanks in advance.
svm= SVC(C= 10, kernel='rbf', gamma= 'scale', decision_function_shape= 'ovo', max_iter=-1, probability=True)
svm_mod= svm.fit(xtr, ytr)
print('yi * alpha_i: \n', svm_mod.dual_coef_)
print('Number of support vectors of each class:', svm_mod.n_support_)

# Output
yi * alpha_i: 
 [[  4.58635603  -0.          -0.          -0.          -0.
   -4.58635603  -0.          -0.          -0.          -0.
   -0.          -0.          -0.39551576  -0.          -0.
   -1.10882587  -0.          -0.          -0.        ]
 [  1.50434163  10.          10.          10.           3.00420504
    0.          10.          10.          10.          10.
  -10.          -3.00420504  -0.         -10.         -10.
  -10.         -10.         -10.         -10.        ]]

Number of support vectors of each class: [1 9 9]

 A: The issue is that you have a multiclass model.  Each column represents a support vector for at least one of the binary sub-classifiers.  So the non-zero property that needs to hold is that each column contains at least one nonzero entry.  From (near the end of) section 1.4.1.1:

The shape of dual_coef_ is (n_classes-1, n_SV) with a somewhat hard to grasp layout. The columns correspond to the support vectors involved in any of the n_classes * (n_classes - 1) / 2 “one-vs-one” classifiers. Each of the support vectors is used in n_classes - 1 classifiers. The n_classes - 1 entries in each row [sic? column?] correspond to the dual coefficients for these classifiers.

There's an example after that; I can't say I found it particularly illuminating (and I think it's suffering the same transposition I noted in the quote), but perhaps you will.

I've had a chance to produce an example notebook.  It uses the iris dataset, with just two features for visualization; in the plots below, class 0 is purple, class 1 is green, and class 2 is yellow.  The support vectors are circled; there are in particular 3 support vectors from class 0, and the first three columns of dual_coef_ are:
Class 0
support vectors    dual coefs
[5.0, 3.0]          2.3,  0.0
[4.5, 2.3]         10.0,  8.0
[5.4, 3.4]         10.0,  5.6

The first dual coefficient for each support vector corresponds to the one-vs-one classifier of class 0 vs class 1, while the second one corresponds to the class 0 vs class 2.  We get evidence for that here: the zero coefficient for the first vector indicates it is outside the margin of the class 0 vs class 2 classifier.


The support vectors for classes 1 and 2 primarily have dual coefficients listed as zero first then something nonzero.  Here are the only exceptions to that:
Class 1:
[5.4, 3.0]        -10.0,   0.0
[5.2, 2.7]         -2.3,   0.0
[4.9, 2.4]        -10.0,   0.0

Class 2:
[5.6, 2.8]         -3.6, -10.0
[4.9, 2.5]        -10.0, -10.0

In the top plots, note the green points within the margin are these three, and in the second plot, the yellow points within the margin are these two.
Finally, below is the last ovo model.  There are obviously many more support vectors, which I won't list.  But note that the class 1 support vectors listed above are all outside this margin, and have zeros as the second dual coefficients; however, the class 2 support vectors above are both on the wrong side of the separating plane (not in the margin, but still misclassified!) and have nonzero dual coefficients in the second slot.

