1
$\begingroup$

I was asked to build a predictive classification model that can predict some types of response. I am interested in 6 classes, however, the total occurence of these 6 classes (out of almost half a million total observations) is only ~2000, and all other responses belong to two huge classes that I'm not interested in. The number of variable is 25. That leads to a problem, since more than 98% of the observation belongs to a two classes, when apply methods like knn or classification tree, 100% of the data will be assigned to those two classes, which makes the prediction meaningless.

I have no idea about how to deal with this problem. I try to reduce the dimension of the data by removing observations with many missing values, but the proportion of the two dominant classes is still about 95%. I would try to study only on the ~2000 observations that belong to "interesting" classes, but that seems to a very bad method...

So anyone can give me some advice about general approach to this problem? I appreciate any help!

$\endgroup$
1

1 Answer 1

1
$\begingroup$

One possibility is to collapse the four rare classes down into one, or set up a model to predict those separately. That may change your problem entirely, however, and not be appealing to you.

Gary King of Harvard University published an excellent paper (http://gking.harvard.edu/files/0s.pdf) on logistic regression for rare events. In this case you oversample the rare events (or collect all rare events) but only a small sample of the more common events. I think this would be a good place to start, but you may ultimately be able to branch out to corrections for multinomial logistic regression. I don't know of resources for rare event multinomial logistic regression, but this may get you started.

$\endgroup$
3
  • $\begingroup$ Thanks! Unbalanced Sampling would be a great idea. I looked at some R documents of packages like 'ROSE' and 'unbalanced' but they can only handle binary response, so maybe I need to do the sampling by myself. Is it pretty strightforwad? Since it seems that I only need to make a new data set with almost even proportion of majority class and rare classes by sampling, or there is something else that I need to consider? $\endgroup$
    – dave2d
    Nov 18, 2015 at 5:55
  • $\begingroup$ And also, after getting the unbalanced sampling data set, it looks like knn is not helpful here because there will be many ties, should I apply classification tree? What I think is doing a tree several times, every time with different unbalanced sampling, and use majority vote of different predicted classes by those trees to determine the final predicted class. $\endgroup$
    – dave2d
    Nov 18, 2015 at 8:10
  • $\begingroup$ @dave2d I would be wary of resampling, mostly because it is very difficult to know just the right amount of resampling needed to correct any undue bias against the minority class, and the bias is only likely to be a problem with very small datasets. If you have a very small dataset, it is difficult to have enough data to tune the amount of resampling. $\endgroup$ Mar 22, 2022 at 9:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.