Running the following code:
from sklearn.svm import SVC model_2 = SVC(kernel='rbf', degree=2, gamma='auto', C=100) model_2.fit(X_train, y_train) print('accuracy (train): %5.2f'%(metric(y_train, model_2.predict(X_train)))) print('accuracy (test): %5.2f'%(metric(y_test, model_2.predict(X_test)))) print('Number of support vectors:', sum(model_2.n_support_))
I get the following output:
accuracy (train): 0.64 accuracy (test): 0.26 Number of support vectors: 55
I also tried with varying degrees of polynomial kernel and got more or less the same results.
So why does it do such a poor job. I've just learned about SVM and I would have thought that a polynomial kernel of 2nd degree could just project these points onto a paraboloid and the result would be linearly separable. Where am I going wrong here?
Reference: The starter code for the snippets in this post comes from this course
Code for generating data:
np.random.seed(0) data, labels = sklearn.datasets.make_circles() idx = np.arange(len(labels)) np.random.shuffle(idx) # train on a random 2/3 and test on the remaining 1/3 idx_train = idx[:2*len(idx)//3] idx_test = idx[2*len(idx)//3:] X_train = data[idx_train] X_test = data[idx_test] y_train = 2 * labels[idx_train] - 1 # binary -> spin y_test = 2 * labels[idx_test] - 1 scaler = sklearn.preprocessing.StandardScaler() normalizer = sklearn.preprocessing.Normalizer() X_train = scaler.fit_transform(X_train) X_train = normalizer.fit_transform(X_train) X_test = scaler.fit_transform(X_test) X_test = normalizer.fit_transform(X_test) plt.figure(figsize=(6, 6)) plt.subplot(111) plt.scatter(data[labels == 0, 0], data[labels == 0, 1], color='navy') plt.scatter(data[labels == 1, 0], data[labels == 1, 1], color='c') ```