Parameter optimization in one-class SVM of LibSVM

Question

I am new to machine learning and SVMs. I have a general question regarding the optimization of parameters in one-class SVM in libsvm in R. I found similar posts but yet not conclusive answer. Can someone tell me, do I need to optimize both c (cost) and nu parameters in one-class SVM?

Actually my preliminary results using e1071 package (libsvm implementation in R) show that predictions (at least in terms of number of positive versus negative classifications) are dependent only from nu and not from c.

Answer 1

The C parameter is used for multiple classification. The nu parameter is used in one-class classification, so as long as you optimize nu you are fine. nu is the upper bound in the fraction of training points. Please check this link

Depending on the kernel function you choose, you will have to optimize gamma too. The hyperplane varies depending on both parameters. If you choose the linear kernel, then there is no need to optimize gamma, as it uses the default value=0 where the gamma value is equal to 1/n (n is the number of features).

To tune your parameters, you can use the tune.svm function:

tuned <- tune.svm(x=yourFeatures, y=yourLabelY, data = yourData, 
              nu =  0.001:0.5,
              gamma = 10^(-2:0), 
              type='one-classification'                 
              );

Then you can use the best values to design your one-class svm model:

model <-svm(x=yourFeatures, y=yourLabelY,type='one-classification', 
            nu = tuned$best.parameters$nu, 
                    gamma = tuned$best.parameters$gamma
        );

This means your nu value will be the best value between 1% and 50%, meaning you will allow up to 50% of outliers (but you want the best value).

Answer 2

nu and C are alternative constants from alternative formulations of the SVM. You have to choose one or the other. If you are using the svm function of e1071 please check the type parameter in http://www.rdocumentation.org/packages/e1071/functions/svm

type="C-classification" which is the default will use the C-SVM formulation which requires the C parameter, which is called cost in the function

type="nu-classification" uses the nu-SVM formulation which requires setting the nu parameter