I am trying to plot the decision boundary of a One Class SVM.

This is a 2 dimensional representation of my training data Train Data

And here the picture of the prediction obtained on the training data


How can you see the values seems to be assigned randomly. In theory I should obtain a ball that contains great part of the data. Another problem is the absence of the decision boundary that according to the code should be plotted in the second figure.

Am I doing something wrong?

Here the code that I have used to generate the results.

DG = DataGenerator()

X, y = DG.J2M()

train_index = list(range(150)) + list(range(250, 500))
pca = PCA(n_components=2)
pca.fit(sk.preprocessing.scale(X[train_index, :], with_std=False))

x_pca = pca.transform(X)

x1 = x_pca[train_index, :]

pl.scatter(x1[:, 0], x1[:, 1], c=y[train_index])
pl.title('train data')

print('Train SVM')
OCSVM = OneClassSVM(kernel='rbf', degree=3, gamma=0.0, coef0=0.0,
                    tol=0.001, nu=.6, shrinking=True, cache_size=200,
                    verbose=False, max_iter=-1, random_state=None)


a1 = x1[:,0].min() - 20
a2 = x1[:,0].max() + 20
b1 = x1[:,1].min() - 20
b2 = x1[:,1].max() + 20

xx1, yy1 = np.meshgrid(np.linspace(a1, a2, 1000), np.linspace(b1, b2,1000))

Z1 = OCSVM.decision_function(np.c_[xx1.ravel(), yy1.ravel()])
Z1 = Z1.reshape(xx1.shape)

y_est1 = OCSVM.predict(x1)
pl.scatter(x1[:,0], x1[:,1], c=y_est1)
pl.contour(xx1, yy1, Z1, levels=[0],
pl.title('prediction of x1')


I have tried including more data in my estimator and using different values of $\nu$ and $\gamma$

print(x_pca.shape) #(500, 2)

OCSVM = OneClassSVM(kernel='rbf', degree=2, gamma=1./p, coef0=0.0,
                    tol=0.001, nu=0.1, shrinking=True, cache_size=200,
                    verbose=False, max_iter=-1, random_state=None)

y_est = OCSVM.predict(x_pca)

pl.scatter(x_pca[:, 0], x_pca[:, 1], c=y_est)

as can bee seen in the figure there is a clear separation between the faulty and the normal data. Despite that I am not able to train a good estimator.

enter image description here


The value of nu is too large. 0.6 means that you allow up to 60% of outliers (at least, asymptotically). You should bring it down to 1%, 0.1% or whatever you expect the novelty rate to be. Also notice that if you want to use gamma=0, then you can just remove it from the list of parameters. gamma=0.0 will be equivalent to gamma=1/n_features.

  • $\begingroup$ Should I include also the faulty data during training? I have tried different combination of $\gamma$ and $\nu$ but I still get strange results. I will soon update the question including the new code. $\endgroup$ – Donbeo Sep 15 '14 at 15:03
  • $\begingroup$ you can but it's better if you can train with clean data only. A big issue I'm seeing (from the plot) is that you are not scaling the data. SVM is very sensitive to that and, on top of that, non anomalies are supposed to have zero mean. So, if you don't scale to zero mean and unitary variance, SVM will not work $\endgroup$ – Bob Sep 15 '14 at 16:29
  • $\begingroup$ Maybe with this last tip I have solved. I have separated data in normal and non normal. Than I have subtracted to all the data the mean of the normal data and I have divided for the std of the normal data. Tomorrow I will try on another dataset and if everything work I will accept the answer. $\endgroup$ – Donbeo Sep 15 '14 at 19:24

You need to tune gamma for the RBF kernel -- a gamma of zero should mean you place a Gaussian on each data point with width of zero.

  • $\begingroup$ Thanks for answering but if gamma is the equivalent of $l$ of the original paper users.cecs.anu.edu.au/~williams/papers/P126.pdf than the default setting for gamma=0.0 in sklearn scikit-learn.org/dev/modules/generated/… is like to use gamma=1/l . Also with gamma=1 I get similar results. I am new to SVM so maybe I am doing something wrong $\endgroup$ – Donbeo Sep 15 '14 at 14:21
  • 3
    $\begingroup$ Just to clarify: gamma=0 is equivalent to gamma=1/n_features. $\endgroup$ – Bob Sep 15 '14 at 14:26

You can only expect a spherical decision boundary if you are using the linear kernel. Since you are using RBF kernel $k$, there must exist a feature mapping $\phi$ such that $k(\mathbf{x},\mathbf{x}') = \phi(\mathbf{x}) \ \cdot \phi(\mathbf{x}')$. In that higher dimensional space associated with $\phi$ the decision boundary is spherical. In case of RBF kernel $\phi$ is infinite dimensional.

Same is true with standard binary SVM. The decision boundary is only a hyperplane in the space associated with $\phi$. Without kernels there is no nonlinearity in SVM and you could only have a linear decision boundary which is very limiting.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.