I have a dataset with two classes and the ratio of the number of instances in two classes is 1:10. I used SVM for classification and obtained a pathetic performance as it classified all the instances in the larger class. Is there a classifier which is shown to perform well in case of biased datasets?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ I don't know if it's possible to give some general advice on this but in any case it's known as the “class imbalance” problem. Using those keywords should allow you to find a lot of relevant material on this site and elsewhere. $\endgroup$– GalaCommented Jun 9, 2013 at 12:27
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
One common approach is observation weighting and/or oversampling.
Simple as it seems, a quick and dirty approach is to create duplicate observations until they are balanced, and re-run your classifier of choice. Alternatively, you can take a sub-sample of the majority class equal to the size of the minority class. Either approach has issues, but can work well as a quick approach.
For a more general approach, look into the topic area of cost sensitive classification. The core reason that generic classifiers have difficulty with imbalanced classes is that they have an implicit assumption that each type of classification error/success is weighed equally.
In other words, your SVM likes the useless solution it provided because it assumed that getting most of the 90% right is equally important to getting the 10% right. There are methods available to augment many classifiers to accept a cost matrix, allowing you to specify some value (usually economic) to associate to each instance of type I, type II errors and successes with an explicit value.
For an overview, consider this
For a paper that constructs an SVM response to these issues see here
