For example, I have a data set contains 100,000 instances. There are only about 5,000 positive instances and negative instances are 95,000. I wish to fit the data using logistic regression or svm. How can I do it? Is this a cold boot problem?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ See: Does an unbalanced sample matter when doing logistic regression? & Does down-sampling change logistic regression coefficients?. $\endgroup$– Scortchi - Reinstate Monica ♦Commented Nov 18, 2015 at 10:08
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
In SVM you can assign a misclassification penalty per class. The most effective way to deal with unbalanced data sets is to increase the misclassification penalty on the minority class. Your class skew is not that large, so this approach will work fine.
This functionality is available in LIBSVM, which is the most popular back-end for all SVM libraries. Hence, you should be able to do this.
answered Nov 18, 2015 at 10:06
$\begingroup$
$\endgroup$
3
Why do you wish to use logistic regresion or svm?
Things like logistic regressions tend to optimize overall performance. Classify everything to negative and you will have a 95% accuracy misclassifying all your “interesting” cases. Which I am guessing is a horrible result for you in this case.
Simply put, you need to optimize something else and as far as I know, logistic regression will is a lost case. However, this is a fairly common problem called “anomaly detection”.
https://en.wikipedia.org/wiki/Anomaly_detection
You will end up optimizing your ROC curve. Which is a complicated way of saying that is going to be a tradeoff between your true positive rate and false positive rate.
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
Or you could check out Coursera and their Machine learning course from Andrew NG. (Week 9)
Edit: removed bad advice.
-
3$\begingroup$ This is bad advice. Do not use a one-class SVM when you have labeled data of both classes. $\endgroup$ Commented Nov 18, 2015 at 10:05
-
$\begingroup$ Thanks for the feedback. I edited the answer instead of removing it because I still think that the rest might be of some help. $\endgroup$– leoszCommented Nov 18, 2015 at 10:25
-
$\begingroup$ Logistic regression is not a lost case. One can use uneven logit or scobit methods. $\endgroup$ Commented Nov 25, 2015 at 21:19