0
$\begingroup$

If I use model that provide output probability in imbalanced case (say ratio between majority and minority class is 100 : 1), I saw that the output probability of data points from majority class is very High (say 99% or so), and much higher than output probability of data points from minority class. The problem is: In case of abnormality detection in banking or in many cases in medical study, we just want to detect the minority class. So I want to increase the output probability of minority class. What can we do in this case? I searched many sources on the internet and papers, but did not see any solutions to this problem. Maybe because people in machine learning mostly care about some metrics like accuracy, then they just apply under/over sampling to improve performance.

Thank you for reading my question.

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ You “just want to detect the minority class”? Just call everything a member of the minority class! Then you will have perfect performance! If this is not an acceptable solution, why is perfect ability to detect the cases of interest not perfect for your work? $\endgroup$
    – Dave
    Commented Aug 15, 2021 at 12:07
  • $\begingroup$ Dave: Maybe the way i said in original post make you feel so extreme about this. The main problem is i just want to improve out put probability of minority class compare to majority class. Moreover. in practice, you can not just predict every thing is in minority class. Since (in banking/medical/marketing), we do not have infinite budget to deal with all cases. $\endgroup$
    – i_love_thu_ha
    Commented Aug 15, 2021 at 12:11
  • $\begingroup$ In other words, you care about both kinds of misclassifications, calling minority classes majority and calling majority classes minority? $\endgroup$
    – Dave
    Commented Aug 15, 2021 at 12:15
  • 2
    $\begingroup$ Harrell discusses marketing in one of his blog posts, among the links below. Why do you (seemingly) want wrong probabilities of membership?stats.stackexchange.com/questions/357466/… fharrell.com/post/class-damage fharrell.com/post/classification stats.stackexchange.com/a/359936/247274 stats.stackexchange.com/questions/464636/… twitter.com/f2harrell/status/1062424969366462473?lang=en $\endgroup$
    – Dave
    Commented Aug 15, 2021 at 12:30
  • 1
    $\begingroup$ See this answer on benefit of using sensitivity and specificity instead of accuracy for small class sizes. stats.stackexchange.com/a/533900/318288 $\endgroup$
    – user318288
    Commented Aug 15, 2021 at 15:27

1 Answer 1

Reset to default
0
$\begingroup$

Use a generative classifier that learns the likelihood of the class of interest. Perhaps start with a Naive Bayes classifier with a uniform prior.

Improve this answer
$\endgroup$
2
  • $\begingroup$ How does a discriminative model not predict adequately? $\endgroup$
    – Dave
    Commented Aug 15, 2021 at 13:00
  • $\begingroup$ A discriminative model perhaps has a higher tendency to learn the relative frequency of occurrences of the target classes. The generative model learns the distribution of each class in isolation and hence a uniform prior puts them on equal footing. $\endgroup$
    – Jayaram Iyer
    Commented Aug 15, 2021 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.