Logistic regression model - AUROC as performance measurement

Question

I am analyzing the probability of default on loans with the aid of a binary logistic regression model. I have therefore fitted the logit of default: $log\dfrac{P(Y=1|\boldsymbol{x})}{1-P(Y=1|\boldsymbol{x})} = \boldsymbol{X\beta}$.

edited

The goal of the analysis is set up a model that can be used to classify borrowers in categories depending on the risk of defaulting. I want to use this model when deciding on setting an appropriate interest rate and also declining high risk borrowers loans.

Clearly my model needs to be able to predict/forecast future risk so it has to be well calibrated.

As a measure of performance, I am unsure whether I can use AUROC in any aspect? As for my understanding is that diagnosis can be made att present time; but I can't tell you anything about the borrower in present time(default or not). Therefore I don't see why my model should be able to discriminate at all.

Does anyone agree or disagree with the fact that I don't need to consider the discriminatory power of my model? can you give my an eyeopener?

Iam trying to understand it with an example:

Suppose I have 2 competing models $A$ with AUC $100$% and $B$ with AUC $50$%

For some subjects of $A$ we predict predict probability of default $s_1 = 0.6, s_2=0.61,s_3 = 0.4,s_4=0.55,s_5 = 0.3,s_5 = 0.2, s_6=0.6,s_7 = 0.59,s_8=0.7,s_9 = 0.1, s_{10} = 0.62$

Say we have some cutoff $0.5$ so that we reject loans with PD $>0.5$. Then we would reject $s_1,s_2,s_4,s_6,s_7,s_8,s_{10}$

why is $A$ preferable to by $B$? it is still the the case that all the rejected $\{s_j\}$ could default, so why is model $A$ better?

Answer 1

Your conceptualization has a lot of problems, and this is symbolized by the use in inappropriate terminology, e.g., using the word multivariate when you have only a univariate $Y$. The goal of credit risk scoring is to estimate credit risk, not to create categories for borrowers. Risk is a continuous measure that should be used continuously (look up lift curves as an example).

Then you confused longitudinal with cross-sectional designs. Most (but not all) risk analyses use cross-sectional data, then envision applying the predictions to a future, similar cohort of subjects. For that setup, measuring predictive discrimination ($c$ index aka AUROC) is a good idea. Rank-based indexes such as $c$ are not optimal (although they are useful and more understandable) when compared to likelihood-based indexes such as pseudo $R^2$.