# How to choose the cutoff probability for a rare event Logistic Regression

I have 100,000 observations (9 dummy indicator variables) with 1000 positives. Logistic Regression should work fine in this case but the cutoff probability puzzles me.

In common literature, we choose 50% cutoff to predict 1s and 0s. I cannot do this as my model gives a maximum value of ~1%. So a threshold can be at 0.007 or somewhere around it.

I do understand ROC curves and how the area under curve can help me choose between two LR models for the same dataset. However, ROC doesn't help me choose an optimum cutoff probability that can be used to test the model on an out-of-sample data.

Should I simply use a cutoff value that minimizes the misclassification rate? (http://www2.sas.com/proceedings/sugi31/210-31.pdf)

Added --> For such a low event rate, my misclassificiation rates are affected by a huge number of false positives. While the rate over all appears good as total universe size is also big, but my model should not have so many false positives (as it is an investment return model). 5/10 coeff are significant.

• It's the relative cost of the two kinds of misclassification together with their probabilities that should determine the cut-off. If you just want to validate the probability model, calculate its AUC or Brier score when applied to the test set. – Scortchi - Reinstate Monica Mar 25 '14 at 16:59
• This might be a good answer: stats.stackexchange.com/a/25398/5597 – Tae-Sung Shin Mar 25 '14 at 17:15
• Also relevant answers here & here. – Scortchi - Reinstate Monica Mar 25 '14 at 17:31
• @Tae-SungShin Thanks for the link. It is helpful. I guess there is not a definite answer to my Q. My model suffers from high number of false positives. – Maddy Mar 26 '14 at 4:06
• @Scortchi Thanks. Using AUC could have been useful if I was comparing 2 different logistic regression models (with extra predictors) but I'm not sure how it helps me in my case. It gives me a total success probability of my model but it doesn't help me chose a cutoff probability. – Maddy Mar 26 '14 at 4:08