0
$\begingroup$

I am working on LibSVM to classify user comments as negative and positive.

I am trying all possible parameter right now however i was not able to find useful information about these parameters

Can you give me more information about them?

In addition so far my tests shows some of them have 0 effect on results

-d degree : set degree in kernel function (default 3)

Tried Degree 1-9 no changes observed. What does it do?

-r coef0 : set coef0 in kernel function (default 0)

Tried coef0 1-9 no changes observed. Again what does it do?

-e epsilon : set tolerance of termination criterion (default 0.001)

I did not change epsilon yet what does it do? If i change what should be the changes range?

-h shrinking: whether to use the shrinking heuristics, 0 or 1 (default 1)

Still testing shrinking but so far no changes. What does it do?

-b probability_estimates: whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)

Did not try probability yet does it change anything?

I am using C# for testing. In addition libSVM.dll gives error if i make it run as multi threading. After first task completed second task always throws bad memory error. Can you verify libSVM is not supporting multi threading?

Ty

Improve this question
$\endgroup$
4
  • $\begingroup$ on multithreaded, have u tried liblinear instead? csie.ntu.edu.tw/~cjlin/liblinear $\endgroup$
    – Jeffrey04
    Commented Oct 16, 2015 at 10:09
  • 2
    $\begingroup$ Have you tried reading the documentation? $\endgroup$
    – Sycorax ♦
    Commented Oct 16, 2015 at 11:40
  • $\begingroup$ @Jeffrey04 ty very much for answer i needed multi-threading for multiple parameter evaluation. you know so i could start multiple tasks to test out multiple parameters. however in single exe when i start task it gives error. multiple exes in different folders works fine though. $\endgroup$
    – Furkan Gözükara
    Commented Oct 16, 2015 at 14:42
  • $\begingroup$ @user777 yes but cant find much info about them $\endgroup$
    – Furkan Gözükara
    Commented Oct 16, 2015 at 14:43

1 Answer 1

Reset to default
6
$\begingroup$

I think you are misunderstanding a lot of things about how to use SVMs.

Most parameters you mention are related to the hyperparameterization of common kernel functions (described in detail in the documentation). By default, LIBSVM uses the so-called RBF kernel: $$\kappa(\mathbf{u},\mathbf{v}) = \exp(-\gamma \|\mathbf{u}-\mathbf{v}\|^2),$$ with $\gamma$ a hyperparameter you must choose.

If you use a default SVC with RBF kernel, you have to choose good values for the misclassification penalty (-C), kernel bandwidth (-gamma) and, optionally, class weights (-wX for class X). To find good values you typically optimize some score function (e.g. cross-validated area under the ROC curve).

LIBSVM itself only offers optimizing cross-validated accuracy via grid search, which has several issues. I recommend to have a look at Optunity, a library designed specifically to automate hyperparameter optimization (I'm the lead dev). Its documentation contains a comprehensive example about tuning a support vector classifier in scikit-learn ($\approx$ LIBSVM), available here.

If you are new to machine learning, I recommend using libraries with a simple API like Python's scikit-learn, instead of using LIBSVM directly. LIBSVM is essentially meant as a back-end for more high-level libraries, and hence has a very terse user interface. For SVM classification, you should particularly have a look at sklearn's SVC documentation.

The other kernel-related parameters (degree, coef0) are entirely irrelevant if you use the RBF kernel.

$\epsilon$ is only really relevant if you're doing regression.

Whether or not you use shrinking (-h) is about using some heuristics in the optimization process, which do not affect the actual resulting model significantly, just the training speed.

-b gives probability estimates, but only if you explicitly ask for them (by default, LIBSVM returns only labels).

LIBSVM itself is not multithreaded as far as I know.

Improve this answer
$\endgroup$
8
  • $\begingroup$ ty very much for answer. I have implemented libsvm to my own c# application. I am going to use it to classify commercial products comments. Assume there is sold iphone and people commented but no comment score. So i will classify those comments as negative and positive. Yes it is true that i don't have too much knowledge about svms as because i don't have that much time at the moment. i am working on multiple issues. I am using grid search for optimization of parameters it is correct. $\endgroup$
    – Furkan Gözükara
    Commented Oct 16, 2015 at 14:37
  • $\begingroup$ ok so just to be sure when using rbf i have to tweak only gama and C, when polynomial gama, C and degree (coef0 not very important right?), when linear only C and when sigmoid only gama and C right? in addition when using Nu_SVC i use nu value instead of C and nothing else changes right? also are there any documentation about what ranges should i try? i mean for example for degree i should try like 1,2,3,4 ? I also have to use C# and it has to be 64 bit supported. it is very sad that libsvm is not supporting multi-threading :( $\endgroup$
    – Furkan Gözükara
    Commented Oct 16, 2015 at 14:39
  • 1
    $\begingroup$ Run svm-train without further arguments to get a documentation dump. For the polynomial kernel you need degree, coef0 and gamma , for sigmoid you need gamma and coef0 (coef0 is quite important in both cases). In practice, don't bother with the sigmoid kernel. It's usually a safe bet to start with the linear kernel and if the performance isn't adequate move on to RBF. If neither of those work well, you typically use another learning method (easy) or manually design a more appropriate kernel function (hard, don't do this unless you really know what you're doing). $\endgroup$
    – Marc Claesen
    Commented Oct 16, 2015 at 15:22
  • 1
    $\begingroup$ LIBSVM doesn't support multithreading because the training of kernel SVM's is almost not parallelizable. Nu-SVM is just a different parameterization, so I wouldn't bother with it. The ranges you should consider depend on a lot of things, including the normalization of your data and the amount of instances, so I can't say. $\endgroup$
    – Marc Claesen
    Commented Oct 16, 2015 at 15:24
  • 1
    $\begingroup$ Finally, the description of your application implies a semi-supervised context (you have scores for some purchases but not others). I recommend learning the basics of that before delving into the specifics of SVM. Essentially, you probably want to weigh data instances with known scores higher than others, but use both labeled and unlabeled instances to build your models. All of this is possible via LIBSVM, but I reiterate that you are likely better off using a higher level library. LIBSVM is really intended for experts, you are probably going to lose time using it for your application. $\endgroup$
    – Marc Claesen
    Commented Oct 16, 2015 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.