sklearn's SVM classification failure

Question

I'm trying to fit a trivial classifier but I'm not sure what am I doing wrong. I'm providing scikit-learn's svm.SVC linear classifier with two samples of X=[[0.], [0.5]] and labels y=[0, 1] and I get a perfect classification but then when I add another tagged sample to X of [0.4], and the corresponding label 1 to y, and try to fit again, the classification fails and I always get a prediction of 1. Why does it fail?

Sample code:

from sklearn import svm
import numpy as np


clf = svm.SVC(C=1, kernel='linear')
X = [[0.], [0.5]]
y = [0, 1]
clf.fit(X, y)

print('coefs: ', clf.coef_)
print('svs: ', clf.support_vectors_)

if np.all(y == clf.predict(X)):
    print('classification worked')
else:
    print('classification failed:')
print('X=', X, ',y=', y, ' ,prediction=', clf.predict(X))

print('\n\n')

X.append([0.4])
y.append(1)

clf = svm.SVC(C=1, kernel='linear')
clf.fit(X, y)

print('coefs: ', clf.coef_)
print('svs: ', clf.support_vectors_)


if np.all(y == clf.predict(X)):
    print('classification worked')
else:
    print('classification failed:')
print('X=', X, ',y=', y, ' ,prediction=', clf.predict(X))

and the output:

coefs:  [[0.5]]
svs:  [[0. ]
 [0.5]]
classification worked
X= [[0.0], [0.5]] ,y= [0, 1]  ,prediction= [0 1]


coefs:  [[0.4]]
svs:  [[0. ]
 [0.4]]
classification failed:
X= [[0.0], [0.5], [0.4]] ,y= [0, 1, 1]  ,prediction= [1 1 1]

Answer 1

There is a balance between the margin width and the misclassification error (depending on the slack variables). Apparently $C=1$ favours the margin width more. If you increase it, forcing it to penalise misclassification more severely, it'll find a better split point.

from sklearn import svm
import numpy as np


clf = svm.SVC(C=1, kernel='linear')
X = [[0.], [0.5]]
y = [0, 1]
clf.fit(X, y)

print('coefs: ', clf.coef_)
print('svs: ', clf.support_vectors_)

if np.all(y == clf.predict(X)):
    print('classification worked')
else:
    print('classification failed:')
print('X=', X, ',y=', y, ' ,prediction=', clf.predict(X))

print('\n\n')

X.append([0.4])
y.append(1)

clf = svm.SVC(C=10, kernel='linear')
clf.fit(X, y)

print('coefs: ', clf.coef_)
print('svs: ', clf.support_vectors_)


if np.all(y == clf.predict(X)):
    print('classification worked')
else:
    print('classification failed:')
print('X=', X, ',y=', y, ' ,prediction=', clf.predict(X))

which yields the following

coefs:  [[0.5]]
svs:  [[0. ]
 [0.5]]
classification worked
X= [[0.0], [0.5]] ,y= [0, 1]  ,prediction= [0 1]



coefs:  [[4.]]
svs:  [[0. ]
 [0.4]]
classification worked
X= [[0.0], [0.5], [0.4]] ,y= [0, 1, 1]  ,prediction= [0 1 1]