ROC curve: how come I have an awful AUC (0.57) with a significant p value (<.001)? I'm trying to identify a cut-off in a sample of values (areas): I'd like to state "if you encounter an area greater than... you can suspect a pathologic condition" or something like this. It's a case-control study, I have a sample of areas belonging to cases and a sample belonging to controls: there is quite an overlap between cases and controls within the smaller areas, but when it comes to higher values of areas, only the case group displays them (image). 
I don't understand why I can't find a good cut-off value to discern between my two populations when the difference between them (calculated with nonparametric Mann-Whitney test) is highly significant (p<.001)...
 A: First let me generate some data
set.seed(123)
a <- rnorm(20, 0, sd=1)
b <- c(rnorm(30, 0, sd=1), rnorm(10, 3, sd=1))
x <- c(a, b)
x <- x-min(x)
x <- x/max(x)
y <- c(rep(0,20), rep(1,40))
plot(jitter(y), jitter(x), ylab='score', xlab='group')


and some plots
library(ROCR)
pred <- prediction(x, y)
auc <- performance(pred, 'auc')
print(auc)
auc_plot <- performance(pred, "tpr", "fpr")
tpr_plot <- performance(pred, "tpr")
fpr_plot <- performance(pred, "fpr")
prec_plot <- perf <- performance(pred, "prec")

par(mfrow=c(2,2))
plot(auc_plot, colorize=T, main="ROC curve")
plot(tpr_plot)
plot(fpr_plot)
plot(prec_plot)


In this case AUC is 0.58375. But ROC is a trade of between true positive rate and false positive rate, since you have such a big overlap, you will never have good trade-off between true positive and false positive value. Therefore, there is no apparent threshold to choose, and in general your model looks horrible. ROC is however not telling the whole story and you can get threshold based on other measures. 0.65 seems reasonable in my case
