Correct conditional expectation via logistic regression but terrible AUC Suppose you have a binary random variable $Y$, and several other random variables $X_1,...,X_p$. Your goal is to "predict $Y$ using $X_1,...,X_p$." So, you go ahead and fit logistic regression, which is trying to estimate $P(Y=1 | X_1,...,X_p)$. However, when you evaluate the results of your logistic regression, you find that the AUC is terrible. 
One possibility is that your model is "bad," i.e. your estimates of the coefficients for $X_1,...,X_p$ are incorrect. Perhaps you did not have enough samples, or the model was specified incorrectly etc. However, it also occurs to me that your model could be "perfect," but simply that the $X_1,...,X_p$ are not "very predictive."
By this, I mean that suppose $P(Y=1)=0.1$, but when you condition on the random variables $X_1,...,X_p$, the (true) probability does not change very much. For example, 
$$0.05\leq P(Y=1|X_1=x_1,...,X_p=x_p) \leq 0.2$$
for all $x_1,...,x_p$.  In other words, $Y$ is almost independent of $X_1,...,X_p$. So, even if the logistic regression model perfectly learns the conditional expectation, the AUC could still be very bad! 
Is there a name for this situation? And is there a way to detect it? How can I tell that my conditional expectations are "correct," even though my predictive ability is so poor? Or am I not even phrasing the problem correctly statistically?
 A: One must first ask if it is a "problem" at all. My viewpoint is that of a chaotician. I would look at the set of $X_1, \ldots, X_p$ and say that set of regressors is too small, and consequently my $n$ is too small too. If I needed to have an AUC of 1, I would get it by having an infinite set of regressors and of samples, and I could deterministically predict the outcome. Just because all of $X_1, \ldots, X_p$ predictors are statistically significant in the logistic model does not mean they comprise a complete set of predcitors (or even a good set at that). You likely (always) only have a subset of the ones you need, and there's always room for more. 
Cancer is a good example, as there are lots of "risk models" that laudably try to predict it. Oversimplifying the background: cancers are caused by a random genetic mutation that destroys a cell's "self-destruct timer"; that mutation is caused by a complex sequence of signalling of transcriptase, reverse transcriptase, etc., moderated by the circulation, the functional status of the immune system, etc.. basically it is as close to a chaotic process as we can come. Practically, when we account for "easy to measure things" like age, smoking status, family history, etc. we are lucky to get an AUC of 0.65. Unless we are able to prize apart a strong genetic or phenotypic predictor, that is as good as we will ever do. 
There are a number of scenarios to describe the above situation. One of the most commonly referred to "omitted variable bias" or, more precisely, "model misspecification" is quite accurate. There are variables, interactions, or alternative expressions that just aren't discovered yet, measured yet, or are practically able to fit with a finite sample size. 
Prespecifying a target AUC is an excellent idea when setting out to make a risk prediction model. I think this is the only basis on which you can call an AUC "good" or "bad". Unrealistic expectations breed faulty data analyses. I've seen healthcare agencies go nearly bankrupt purchasing software which touts superior predictiveness to fellowship trained radiologists. When I read about the operating characteristics of these models, I found it just wasn't true. Retrospectively, the software only increased false positives and so cost more than just money, but stress, increased exposure of at-risk women to radiation via mammography, and unnecessary medical procedures (confirmational biopsy).
In prespecifying your target AUC, the reviewer of your research can check their expectations at the door: is this a clinically meaningful AUC? Is it a substantial improvement over existing technology? Is it in fact believable given other research I have encountered?
