Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

I applied SVM to perform the classification against several data sets. It turns out that the performance metric of recall is pretty bad for one data set. It has recall around 50% while other data sets have recall around 80%. For this kind of scenario, what are the possible approaches that are available to improve the recall? Besides, why some data sets can have a poor performance in terms of recall? How to analyze this kind of problem?

share|improve this question
Sample sizes by class will have a big impact. You will get the same validity and generalization issues that arise when using stepwise regression methods. – DWin Mar 4 '12 at 19:37

2 Answers

up vote 2 down vote

Are you sure that your data-set is balanced (# of negative examples close to # of positive examples)? SVM is very limited when the dataset is imbalanced...

Low recall means you have many false negatives, which might mean that you have more positive examples than negatives?

share|improve this answer
Roronoa, yes, you are right, it is imbalanced data set. For this kind of data set, what's the best approach to classify them? – bit-question Feb 3 '12 at 14:36
Well, if you can't balance your data, you will need to tune your parameters. There're weight parameters for each class that you can specify (so by increasing the weight in one class, you basically increase the penalty of classifying that class wrong) - I don't think there's a magic formula for choosing the parameters (I might be wrong). There's also a cost parameter, which adds penalty on both classes (of course, if you increase that too much, you'll probably over-fit) – Roronoa Zoro Feb 3 '12 at 15:36
I know you can set these parameters in the libsvm library. I'm not sure about the one you're using... – Roronoa Zoro Feb 3 '12 at 15:37
up vote 0 down vote

SVM algorithms sometimes have a difficult time of determining a good value of the offset.

You should construct an ROC curve (http://en.wikipedia.org/wiki/Receiver_operating_characteristic) from the scores that the SVM algorithm outputs. (Sort the scores and use each as a positive/negative classification threshold.)

Then, based on the ROC curve, set the new offset parameter as the score that best balances the precision/recall.

Alternatively, you could use one of the SvmPerf (http://svmlight.joachims.org/svm_perf.html) options that allows you to directly optimize precision and recall.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.