2
$\begingroup$

I am using LibSVM (3.18) as an implementation of SVM. But every time when I'm predicting the result, it's giving zero.

I am following these instructions:

  • I have CSV file (+50K lines), Most of data in column (target) is zeros, the other values are between 1-10.
  • I convert csv file to libsvm data by selecting this column as label.

  • When I Scale Data, I use these parameters

    $ svm-scale -l 0 -u 1 data.cv>scaled.data

  • As I have a huge file, I use Subset.py.

  • When I finish all the steps and apply predict. I got good result of accuracy.

    $svm-predict scaled_data.csv model.train data.predicted

    Accuracy = 94.28%

but the file I get (data.predicted) contains only zeros.

What does this mean statistically ?

Is it tricky to predict this kind of data ? Is there any way to solve this problem ?

Improve this question
$\endgroup$
4
  • $\begingroup$ Assuming the positive label is +1 you should do svm-train -w+1 100 -c 0.03125 -g 0.0078125 -v 5 -q. Note that you might need to optimize c again. $\endgroup$
    – Marc Claesen
    Apr 20, 2014 at 12:08
  • $\begingroup$ Perfect! One more thing, by mentioning -c, i run grid.py on small data that extracted using subset. how accurate is subset in this case ? I mean, if we get small data, we might find only zeros in this data as we have a lot of negative points $\endgroup$
    – user3378649
    Apr 20, 2014 at 12:12
  • $\begingroup$ If you have no instances of the positive class in a subset you cannot learn anything. libsvm will complain when this happens. $\endgroup$
    – Marc Claesen
    Apr 20, 2014 at 12:13
  • $\begingroup$ grid.py is taking a long time (+24 hours) when i apply on the whole file, that's why I used subset to accelerate the process $\endgroup$
    – user3378649
    Apr 20, 2014 at 12:15

1 Answer 1

Reset to default
1
$\begingroup$

You probably have very few positives compared to negatives. This is called an imbalanced setting. In this case, predicting everything as negative will yield high accuracy even though it is useless.

If you indeed have an imbalanced data set, you can improve your models by assigning a higher misclassification to positive instances during training. In libsvm you can do this using the -w flag.

A good heuristic to assign class weights is to use the following equation: $$C_{pos} \times n_{pos} = C_{neg} \times n_{neg}.$$ Ex.: if you have 1 positive and 100 negatives, $C_{pos}$ should be $100 C_{neg}$.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks ! That's very very useful. at which step should I do this ? (I mean: when I scale, predict or cross-validate data ..) $\endgroup$
    – user3378649
    Apr 20, 2014 at 11:41
  • $\begingroup$ You can assign misclassification penalties per class while training models using svm-train. $\endgroup$
    – Marc Claesen
    Apr 20, 2014 at 11:44
  • $\begingroup$ "I don't lack data, but the phenomena I am doing a study about is rare", I am doing statistics now about the data I have; 98% of the data is zero. Is it accurate to use the factor this way. $\endgroup$
    – user3378649
    Apr 20, 2014 at 11:52
  • $\begingroup$ I couldn't found "-w" flag, but I found "-wi weight" : set the parameter C of class i to weight*C, for C-SVC (default 1) $\endgroup$
    – user3378649
    Apr 20, 2014 at 11:56
  • $\begingroup$ @user3378649 that is the flag you need. Replace i by the label of the positive class. $\endgroup$
    – Marc Claesen
    Apr 20, 2014 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.