One class classification with libsvm. Accuracy results in 0%

Question

A quick recap for what I want to do, I want to determine if a text is written by the same author or not. Thus I use one-class classification.
In my training set (18 samples), it looks like this (for simplifying, I used x as data value):

1 1:x 2:x "until" 200:x
1 1:x 2:x "until" 200:x

In my testing set (3 samples), it looks like this (for simplifying, I used y as data value):

1 1:y 2:y "until" 200:y

For data preparation (training and testing set), I set upper and lower scaling limit to +-1

-l -1 -u 1

For training, I use svm_type is one class svm, kernel type is sigmoid. Yet the accuracy is 0%

optimization finished, #iter = 13
obj = 22.901769047004553, rho = 5.476401914859387
nSV = 11, nBSV = 6
Accuracy = 0.0% (0/21) (classification)

Can someone show me what I did wrong here?
Thank you very much.

Answer 1

As per the comments, I suspect an issue with scaling. Another possibility is poor choice of hyperparameters, in the case of one-class SVM these are $\nu$ and kernel parameters.

Scaling

When using SVM's it is appropriate to scale your data, which you have done. In scaling, however, it is important to use the same coefficients to scale both training and testing set. This is explained in this practical guide to SVM classification (see 2.2 Scaling).

If you use different coefficients, your training and test sets become incompatible. The smaller your sets are, the larger this incompatibility may be (you are quite prone to this).

I am going to guess you scaled like this:

svm-scale -l -1 -u 1 train.txt
svm-scale -l -1 -u 1 test.txt

This is wrong! The scaling tool in LIBSVM internally computes coefficients based on the minimum and maximum per feature. Clearly these may differ between data sets (the larger, the less likely the difference will be substantial).

To ensure you use a single set of coefficients, use the following commands:

svm-scale -l -1 -u 1 -s coefs.txt train.txt
svm-scale -r coefs.txt test.txt

This saves the coefficients computed based on the training set and reuses them to scale the test set. This way they are compatible.

Hyperparameters and choice of kernel

When using SVM (any formulation), it is important to use optimal values of the hyperparameters. You used the sigmoid kernel (why?), which has the following kernel function: $$\kappa(\mathbf{x}_i,\mathbf{x}_j) = \tanh(\gamma \mathbf{x}_i^T\mathbf{x}_j + c_0)^{d}$$

This is quite a complex kernel functions with 3 tuning parameters (that is a lot). It is known to cause numerical issues. I suggest considering the RBF kernel instead, which has one tuning parameter and no numerical problems.

Since you have used one-class SVM with a sigmoid kernel, you have 4 parameters ($\nu$, $\gamma$, $c_0$ and $d$). Tuning all of these is going to be a hassle and will most definitely cause an overfit because your data sets are tiny. Yet another reason to get rid of the sigmoid kernel.