1
$\begingroup$

I have a long tailed distribution with many classes, and the num of samples per class is

[38K, 12K, 8K, 6k, 4K,4K, 2K, 800, 600, 400,400,300,280,180,180,120, 85, 70, 60, 60, 60, 60, 50]

I tried to train a multiclass classifier and got pretty good results AUC for the 3 largest class, but very poor results for all the other classes. How can I address this type of classification? Thanks!

Improve this question
$\endgroup$
2
  • $\begingroup$ If you treat all prediction errors as equally costly, that is the sort of outcome you might expect. The benefit of a few more correctly predicted values actually in the smaller classes will often be outweighed by the cost of a larger number of correctly incorrectly predicted values actually in the bigger classes $\endgroup$
    – Henry
    Commented Nov 30, 2022 at 14:08
  • $\begingroup$ Related and perhaps worth a read $\endgroup$
    – Dave
    Commented Jul 31 at 13:56

1 Answer 1

Reset to default
1
$\begingroup$

Use a multiclass probabilistic prediction method, i.e., one that outputs a predicted probability for a new instance to belong to each class. Assess these using proper scoring rules.

In your situation, usually these probabilities will be high for the first few classes and low for the others (unless there are predictors that clearly indicate that an instance belongs to one of the rarer classes, e.g., a biopsy result from a cancer test). And that is precisely as it should be: even conditional on predictors, an instance simply has a higher probability of belonging to a majority class, rather than to a minority one.

Subsequent to this, you can leverage these probabilistic predictions together with costs to come to decisions, by adjusting thresholds. These decisions should take the predictions into account, but also the wider context. If there is a low-but-still-elevated probability for an instance to belong to a particular minority class, this may be a reason for one action (e.g., a deeper analysis, or a background check), even if the highest probability is for one of the majority classes. Similarly, if an instance has high predicted probabilities for two majority classes, you might want to treat it differently from an instance that has a high prediction for only one class. And so on.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.