5
$\begingroup$

The $R^2$ of a model measures how well a model fits the data and is a measure of the shared variation between two (or more) variables. Its equivalent measure for logistic regression is the pseudo-$R^2$. A pseudo-$R^2$ is sometimes presented alongside the area under the receiver operator characteristic (ROC) as a measure of a model's predictive accuracy.

I'm curious as to whether there is any straightforward relationship between these two metrics. Does a model with a higher pseudo-$R^2$ necessarily have a larger AUC ROC? Are there any situations where a model can have a low pseudo-$R^2$ but a high AUC ROC? It seems intuitive that the two measures are necessarily correlated, but I've been wrong many times in the past.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

The AUC is scale independant. It is solely based on ranks. If you multiply all the probabilities outputed by your logistic regression by the same factor $\lambda\in(0,1]$, the AUC will remain the same. Note that as $\lambda\rightarrow0$ the pseudo $R^2$ will decrease (possibly becoming negative).

So you can have a low pseudo $R^2$ but a large AUC.

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ AUC takes some getting used to if you are accustomed to rsq or pseudo-rsq. E.g., an rsq of .05 (5% of the way from 0.00, useless prediction, to 1.00, perfect prediction) will typically be paired with an AUC near .65 (30% of the way from .50, useless prediction, to 1.00, perfect prediction). $\endgroup$
    – rolando2
    Commented Oct 22, 2015 at 15:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.