1
$\begingroup$

I currently have data where there are 5 labels, 1,2,3,4,5 for my $Y$ variable and a set of associated predictors $X$.

The problem is, I have around $10000$ observations with label $1$ and around only $100$ for label 2, $30$ for label 3, and finally down to $10$ for label 5. I am wondering what is a good way to calculate the expected value to predict for a label. Would it be wise to compute probabilities for each class label, then apply the probabilities into a weighted sum? What would someone do to handle this?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/283170/… $\endgroup$
    – Tim
    Commented Dec 7, 2017 at 14:00
  • $\begingroup$ Are you asking a statistical or decision making question? For statiatical modeling, Work with probabilities. For decision making (actually assigning classes to new data) you need to quantify the various costs if misclassification. $\endgroup$
    – Matthew Drury
    Commented Dec 7, 2017 at 16:48
  • $\begingroup$ Can your labels be ranked? $\endgroup$
    – Todd D
    Commented Dec 7, 2017 at 16:48

1 Answer 1

Reset to default
2
$\begingroup$

Oversampling

One approach would be to use oversampling, which aims to generate artificial instances of minority classes based on the data available.

Adjusted Cost Function

The class imbalance impacts the training and evaluation of a classification model. In your case, using uniform costs for each type of misclassification might result in the classifier simply predicting every candidate instance as being of the majority class. This will score well if you're using cost insensitive evaluation measures like accuracy, but in fact, the patterns associated with the minority classes will not be considered by your model.

By defining a cost matrix for your problem and using it in the cost function of a classifier (for example in an SVM or with kNN) can help mitigate the negative effects of the imbalance. In this answer to a question on the topic I've explained how misclassification cost with a cost matrix can be used in evaluating a model.

To the research!

Imbalanced data is a topic that is subject to quite a bit of research in the field. I've heard some interesting talks about the topic during a workshop this year, maybe you can find interesting material in the proceedings. Here you might find other, more recent, approaches.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.