I have a question of choosing the threshold of an imbalanced data. In my data set, there are 90% of the data belongs to class 0 in the dependent variable, and only 10% belongs to class 1.

If I fit a single variable logistic regression, the estimated probability doesn't reach 1 at all. (not a S-curve but just a slightly curved line at the bottom)

If I fit a multiple logistic regression, and I want to achieve 0.8 sensitivity, the threshold has to be as low as 0.05. I don't think this threshold is reasonable. Is it because my data is imbalanced?

  • $\begingroup$ if you only aim for sensitivity, you can make it $1$ if you choose your threshold as $0$, and classify everything as positive class; which is bad of course. This is why you think your threshold is not reasonable. How about tuning your threshold for getting the best F1, which is more robust? $\endgroup$ – gunes Apr 7 at 10:44
  • $\begingroup$ Hmm..my objective is not to maximise sensitivity but to achieve at at around 0.8, that's why I don't use threshold of 0 directly. $\endgroup$ – wong bowie Apr 7 at 15:32
  • $\begingroup$ As mentioned, if I want to achieve 0.8 sensitivity, my threshold has to be very low. I've also tried to code my dependent variable reversely (1 becomes 0 and 0 becomes 1), so I have 90% of the observations belongs to class 1 instead. Then, even if my threshold is as high as 0.8, I still have a very high sensitivity. Therefore, I reckon the threshold choice is somehow affected by the balance of the data as well? Is that right? $\endgroup$ – wong bowie Apr 7 at 15:35
  • $\begingroup$ Yes, threshold depends on the balance of dataset. Split your dataset into train/test and tune your threshold to get 0.8 sensitivity, and use that threshold on your test set, and see if it is still 0.8. However, as I said, having 0.8 sensitivity w/o other metrics doesn't mean much to me. $\endgroup$ – gunes Apr 8 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.