how to solve the problem that the positive instances are much less than negative instances in dataset? For example, I have a data set contains 100,000 instances. There are only about 5,000 positive instances and negative instances are 95,000. I wish to fit the data using logistic regression or svm. How can I do it? Is this a cold boot problem?
 A: In SVM you can assign a misclassification penalty per class. The most effective way to deal with unbalanced data sets is to increase the misclassification penalty on the minority class. Your class skew is not that large, so this approach will work fine.
This functionality is available in LIBSVM, which is the most popular back-end for all SVM libraries. Hence, you should be able to do this.
A: Why do you wish to use logistic regresion or svm?
Things like logistic regressions tend to optimize overall performance. Classify everything to negative and you will have a 95% accuracy misclassifying all your “interesting” cases. Which I am guessing is a horrible result for you in this case.
Simply put, you need to optimize something else and as far as I know, logistic regression will is a lost case. However, this is a fairly common problem called “anomaly detection”.
https://en.wikipedia.org/wiki/Anomaly_detection
You will end up optimizing your ROC curve. Which is a complicated way of saying that is going to be a tradeoff between your true positive rate and false positive rate.
https://en.wikipedia.org/wiki/Receiver_operating_characteristic
Or you could check out Coursera and their Machine learning course from Andrew NG. (Week 9)
Edit: removed bad advice.
