# What's the measure to assess the binary classification accuracy for imbalanced data?

Now I have binary classification problem with positive samples roughly 100 times the number of negative samples. In this case the normal accuracy measure (predict == label) is not a good measure. What other measures there are? Is precision, recall for negative sample fine or F-1 measure the best? If the model is a probability model, is AUC (Area under curve) a good measure?

Any method that uses arbitrary cutoffs and dichotomizes continuous information such as probability of class membership is problematic. And classification accuracy is an improper accuracy scoring rule, being optimized by the wrong model. Concordance probability ($c$-index; ROC area) is a measure of pure discrimination. For an overall measure consider the proper accuracy score known as the Brier score or use a generalized likelihood-based $R^2$ measure.
• Brier score has been used for very imbalanced data since 1951. It's what the US Weather Service uses for judging the accuracy of rainfall forecasts. I don't think the precision, recall, F1 are proper scoring rules. You can supplement the Brier score with the $c$-index (concordance probability; ROC area) which require no thresholding. – Frank Harrell Jul 26 '15 at 17:50
• Forecasts of rare events have the "right" effect on the mean, i.e., mean predicted probability of the event = overall proportion of events. The Brier score works no matter what the prevalence of events. For a measure of pure discrimination, the $c$-index (AUROC) has an interpretation that in fact is completely free of prevalence. – Frank Harrell Jun 7 '17 at 12:26