I have a training set of over a 100,000 points that is used to train a Logistic Regression Classifier (logit, since response is binary). The model is testing/fitted on a test set of 20,000 items. The test set is totally independent.

The ROC AUC value for this model is 0.85 which suggests that this is a good model. But I was not convinced. I picked a threshold $0.5$ (i.e., its classified positive if the model response $> 0.5$, negative if model response $< 0.5$).

At this threshold, I get the confusion matrix:

Confusion Matrix and Statistics

Prediction     0     1
         0 33307   679
         1     0     0

               Accuracy : 0.98            
                 95% CI : (0.9785, 0.9815)
    No Information Rate : 0.98            
    P-Value [Acc > NIR] : 0.5102          

                  Kappa : 0               
 Mcnemar's Test P-Value : <2e-16          

            Sensitivity : 0.00000         
            Specificity : 1.00000 

So my question is, how good is the model if it is unable to predict a 'positive' class at 0.5 threshold?

My guess would be that the threshold of the model for labelling 'positive' is not $0.5$ in this case. Is this intuitive and make sense? Clearly the ROC AUC value is very high, which means that it does have a good TPR rate at lower thresholds.

  • 2
    $\begingroup$ Why threshold 0.5 should be used? Why do not use estimated probabilities and take a decision at point where you get needed lift or if you have a profit function, at point of profit maximization? $\endgroup$
    – Analyst
    Commented Aug 20, 2015 at 4:15
  • $\begingroup$ It may well be perfect. If the density of the minority class (weighted by it's prior probability) is lower everywhere than the weighted density of the positive class, then the Bayes optimal decision rule assigns all patterns to the majority class (see stats.stackexchange.com/questions/539638/…). If this is not an acceptable answer, it means the misclassification costs are not equal and you need to look at cost-sensitive learning (which can be implemented by altering the threshold for logistic regression). $\endgroup$ Commented Mar 6 at 18:14

1 Answer 1


Threshold 0.5 is arbitrary, as is every threshold. Logistic regression produces probabilities and these can be used to make an decision for classification purposes if that is needed.

For practical purposes in telemarketing campaign I recently analyzed it was decided that cutoff at 0.2 would suffice when at sample level response rate for our offer was at 0.09. At that decile lift was over 2 and we got enough target population for this to be profitable.

  • $\begingroup$ Is there a statistical technique I can use to determine what the threshold value is? $\endgroup$
    – masfenix
    Commented Aug 20, 2015 at 21:47
  • $\begingroup$ @masfenix no there is no such thing. But perhaps you could use combination of various metrics. I like directly assigning costs and benefits to various misclassifications and correct classifications. $\endgroup$
    – Analyst
    Commented Aug 25, 2015 at 4:44

Not the answer you're looking for? Browse other questions tagged or ask your own question.