I currently have data where there are 5 labels, 1,2,3,4,5 for my $Y$ variable and a set of associated predictors $X$.

The problem is, I have around $10000$ observations with label $1$ and around only $100$ for label 2, $30$ for label 3, and finally down to $10$ for label 5. I am wondering what is a good way to calculate the expected value to predict for a label. Would it be wise to compute probabilities for each class label, then apply the probabilities into a weighted sum? What would someone do to handle this?

  • 1
    $\begingroup$ See stats.stackexchange.com/questions/283170/… $\endgroup$ – Tim Dec 7 '17 at 14:00
  • $\begingroup$ Are you asking a statistical or decision making question? For statiatical modeling, Work with probabilities. For decision making (actually assigning classes to new data) you need to quantify the various costs if misclassification. $\endgroup$ – Matthew Drury Dec 7 '17 at 16:48
  • $\begingroup$ Can your labels be ranked? $\endgroup$ – Todd D Dec 7 '17 at 16:48


One approach would be to use oversampling, which aims to generate artificial instances of minority classes based on the data available.

Adjusted Cost Function

The class imbalance impacts the training and evaluation of a classification model. In your case, using uniform costs for each type of misclassification might result in the classifier simply predicting every candidate instance as being of the majority class. This will score well if you're using cost insensitive evaluation measures like accuracy, but in fact, the patterns associated with the minority classes will not be considered by your model.

By defining a cost matrix for your problem and using it in the cost function of a classifier (for example in an SVM or with kNN) can help mitigate the negative effects of the imbalance. In this answer to a question on the topic I've explained how misclassification cost with a cost matrix can be used in evaluating a model.

To the research!

Imbalanced data is a topic that is subject to quite a bit of research in the field. I've heard some interesting talks about the topic during a workshop this year, maybe you can find interesting material in the proceedings. Here you might find other, more recent, approaches.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.