3
$\begingroup$

I recently stumbled upon a logistic regression model with four predictors $x_1$ $x_2$ $x_3$ $x_4$, having an AUC = .800 (implemented in R glm(formula, family = binomial)).

$$ ln(\frac{p}{1-p}) = b_0 + b_1 x_1 + b_2 x_2 + b_3 x_3 + b_4 x_4 $$

When a fifth predictor $x_5$ was included in the model, AUC decreased by .005 (AUC = .795). The regression coefficient $b_5$ of $x_5$ was significant with p = .01.

In this example, my understanding of the significant regression coefficient $b_5$ is, that when controlled for all other predictors $x_1$ $x_2$ $x_3$ $x_4$, predictor $x_5$ is able to predict the criterion $ln(\frac{p}{1-p})$ with a type I error rate of $\alpha = .01$. However, this notion conflicts with the fact that AUC decreases, because if $x_5$ were to be independently predicting the criterion, I'd have expected AUC to increase.

There must be some mistake in this line on reasoning. Can anybody enlighten me please? (I saw this question but its answer was not very informative.)

$\endgroup$
  • $\begingroup$ How you calculated the AUC? $\endgroup$ – Deep North Feb 17 '18 at 13:03
4
$\begingroup$

The concordance probability ($c$-index; AUROC) is a convenient and easy to understand measure of pure discrimination ability. It is not intended to be the gold standard as it doesn't use the full information that the log likelihood function does. In particular it does not properly reward extreme predictions that are "right". So this can happen. Concentrate on measures that are functions of the log-likelihood such as pseudo $R^2$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.