I'm not certain I understand how sklearn's Linear SVC works. I had assumed that it would find an optimal hyper-plane to divide one class from another.
I tried to recover the separating hyper-plane from the following example in the docs (https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn.svm.LinearSVC.score):
from sklearn.svm import LinearSVC from sklearn.pipeline import make_pipeline from sklearn.preprocessing import StandardScaler from sklearn.datasets import make_classification X, y = make_classification(n_features=4, random_state=0) clf = make_pipeline(StandardScaler(), LinearSVC(random_state=0, tol=1e-5)) clf.fit(X, y) clf.score(X, y)
and, found the score was
I then tried to extract the dividing hyper-plane and recover this result. I did it the following way:
A = clf.named_steps['linearsvc'].coef_ b = clf.named_steps['linearsvc'].intercept_ C = np.dot(A,X.transpose()) + b C = C[0,:] # This gives C the shape of y
I then tried to rescale
C so that all positive responses correspond to +1 and all negative responses correspond to 0. This was to make
y. I did this as follows, renaming
y_calc = (C > 0).astype(np.int) clf.score(X, y_calc)
I expected to get a perfect score by using
y_calc as the 2nd argument to
clf.score. I did not. The score I got was
This doesn't make sense. I eventually found that if I added a margin, I could get the perfect score I expected:
C1 = C + 2.5e-1 y_calc = (C1 > 0).astype(np.int) clf.score(X, y_calc)
I don't understand this. I found 2.5e-1 by guessing. I don't see where it came from. The specified tolerance in LinearSVC was 1e-5, not 2.5e-1.
Why did I need to add 2.5e-1 to C, in order to get the classifications from clf.score mach those calculated directly from the SVC calculated hyper-plane?
I think the issue lies in the use of StandardScaler() within make_pipeline. Possibly, I need to extract the standard scaling.