Libsvm one-class svm: how to consider all data to be in-class

Question

I am using Libsvm for Matlab.

I would like to construct the model for a full circumscription of all training data (in the higher SVM-space). For this I assume all my training data is correct and has no outliers.

I generate random distributed data (which is likely to resemble my real-world data) and train an one-class SVM for it. When I predict the labels of that same data almost all the data points that are used as Support Vector are also considered to be outside the class. Is that correct behavior?

How can I construct a model for a SVM which considers all data to be in-class?

The following code gives an example. In the resulting scatterplot the blue circles are all the data points, the reds are support vectors used by the model and the green circles are for points that are outside. So, the empty red circles are support vectors but out-of-class.

I have tried to adjust the nu parameter (-n 0.5 as default), but that only changes the ratio of data points / support vectors. The support vectors are still most out-class.

data = normrnd(0,1,1000,2);
labels = ones(length(data),1);

% Construct one-class SVM with RDF kernel (Gaussian)
model = svmtrain(labels, data, '-s 2 -t 2');

% Use the same data for label prediction
[predicted_labels] = svmpredict(labels, data, model);
inside_indices = find(predicted_labels > 0);

figure; hold on;
% Scatterplot of all data, blue circles
scatter(data(:,1), data(:,2), 30, 'blue');

% Scatterplot of all support vectors, small red circles
scatter(model.SVs(:,1), model.SVs(:,2), 20, 'red');

% Scatterplot of all data inside the one-classs, small green circles
scatter(data(inside_indices,1), data(inside_indices,2), 10, 'green');

The resulting scatterplot:

Edit: I may have found a solution; the LIBSVM Tools contains an extension for "Support Vector Data Description", for "finding the smallest sphere containing all data": http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/#libsvm_for_svdd_and_finding_the_smallest_sphere_containing_all_data

Edit2: Using the SVDD tool does make make any difference. I train with -s 5 but still I only get around 50% accuracy for the same data set.

My question still holds; How can I describe all data with an one-class SVM?

Answer 1

If you set the nu parameter to very small (-n 0.001) and set the gamma to small as well (-g 0.001) you will get almost all your training data being in your class.

Answer 2

I tried your code. For one-class-SVM, the penalty determined by a parameter nu. The default value is 0.5. That is what gives you this plot. If you set it to be 1, all the points are classified as one class. Setting C does not affect the output because C is not relevant to one-class-SVM.

Answer 3

I know it's a bit late but anyway my contribution to your question is this:

You are using a gaussian kernel to train your one-class svm. This may mislead you for making deductions about the support vectors presence. If you just modify your example and use a linear one-class svm (always better explained in linear case) you will see that your sv are equally distributed in all the space. (I am not suggesting that linear is the best choice, just that are more straight forward to be be explained).

As you have defined your question is that all the data points that are used as Support Vector are also considered to be outside the class so your concern is why the majority of sv is outside your class? My guess is that your data is quite dense and in the attempt to model the boundary a lot of your outliers are required for this task. If you make another experiment with sparser data I think that wont be such a problem.

Answer 4

I changed the colors:

green -> training set,

red-> SV,

blue->in class data

Also I added the nu value into svmtrain parameters

model = svmtrain(labels, data, '-s 2 -t 2 -nu 0.1');

optimization finished, #iter = 484
obj = 723.098765, rho = 15.666491
nSV = 112, nBSV = 88
Accuracy = 90% (900/1000) (classification)