1
$\begingroup$

My aim is to predict default probabilities over the whole life-time (max. 36 month) of a loan with the information given at the issuance of the loan. So the aim is to determine $P(T=t| T \geq t,x)$. Therefore I have two approaches:

Apporach 1:

Since the loan can only default at discrete time points - the month end - I used a discrete hazard model, which applies a single logistic regression to estimate the life-time prob. of default as follows: $P(Y_{it}=1| T \geq t,x)=h( \alpha_{0t}+ X_i^T \beta))=\frac{exp(\alpha_{0t} + X_i^T \beta)}{1+ exp(\alpha_{0t} + X_i^T \beta)}$

So the only time-varying coefficient is the time-specific intercept and the only time variying variable is the time-factor. All other coefficients and variables are fixed.

To estimate the model above, one needs an "augmentet" model matrix, where for each observation one creates as many rows as the loan "survived". Only the time-factor is varying such that in the end one obtains a model matrix:

$ X_i= \begin{bmatrix} 0 & 1 & x_{i1} & x_{i2} & ... & x_{ik} \\ 0 & 2 & x_{i1} & x_{i2} & ... & x_{ik}\\ \vdots & \vdots &\vdots & \vdots &\vdots & \vdots \\ 1 & t_i & x_{i1} & x_{i2} & ... &x_{ik} \\ \end{bmatrix} $

wehre $i$ denotes the loan , the first row the default indicator and the second row the time-factor. Then one fits one logistic regressoin to the augmented dat-matrix and gets an estimate of the discrete hazard. Further details are described in https://www.researchgate.net/profile/Moritz_Berger2/publication/316098728_Semiparametric_Regression_for_Discrete_Time-to-Event_Data/links/592c336fa6fdcc44435fe3d1/Semiparametric-Regression-for-Discrete-Time-to-Event-Data.pdf

Apporach 2:

Estimate 36 separate logistic regressions using a different model matrix for each regression: For each month a separate default indicator $Y_{it}$ is created, which is one if the loan $i$ defaults in month $t$ and is zero otherwise. The model matrix at each month-regression $t$ consists only out of the loans, which had survived until $t-1$. The model is therefore given by: $ P(Y_{it}=1|Y_{it-1}=0)=\frac{exp(X_i^T \beta_t)}{1+ exp( X_i^T \beta_t)} $

In this model, all variables are fixed but the coefficients vary from time to time regression. So for the first regression, the dependent variable is the default in $t=1$ and only those obs. that survived until $t=1$ are regarded. In the second regression, only those obs. that have survived until $t=2$ are regarded and the dependent variable is default in $t=2$ and so forth.... So the number of obs. reduces in each regression. More details can be found in: https://www.researchgate.net/publication/23522944_In_Search_of_Distress_Risk

Results so far:

Approach 2 yields ROC curves with an AUC close to 1 and McFadden Pseudo Rs of 0.9 for the first regression and 0.7 for the 36. regression - which seems pretty unrealistic. Also some predictors do have unintuitive signs and are insignificant.

Approach 1 semms more realistic with an AUC of 0.7 and a pseudo R of 0.05.

Do you have any guess why Approach 2 outperforms so heavily?

(My guess is that there are some statistical properties which inflate the pseudo $R^2$ and the AUCs...)

$\endgroup$
  • $\begingroup$ My guess is that the 2nd approach is massively overfit. If I understand this correctly, the difference between the approaches is that the second include interactions between every time-factor and every other variable, so it has about 36 x more parameters. $\endgroup$ – The Laconic Aug 18 '17 at 23:05
  • $\begingroup$ Or it's possible I don't understand the second approach. The second approach smells fishy, but I don't really want to read the paper right now. $\endgroup$ – The Laconic Aug 18 '17 at 23:08
  • $\begingroup$ @ The Laconic: Thank you for your reply! Regarding your understanding of the second apporach. You're right: There are 36 x more parameters. Also, for the first regression, only those observations are included, which have "survived" until t=1, with dependent variable default in t=1. For the second regression, only those obs. which have survived until t=2 are used and the dependent variable is default in t=2 and so forth...hopes this clarifies? $\endgroup$ – Jogi Aug 19 '17 at 8:29
  • $\begingroup$ But is it that ALL of the observations which have survived until t=2 (for example) are included? What I'm confused about is whether the second approach is really identical to the first, but includes all interactions between the time-factor and every other variable, or whether the second approach is something different. $\endgroup$ – The Laconic Aug 19 '17 at 14:44
  • $\begingroup$ I reread what you wrote, and I still think it's the former---that they're the same approach and the "36 separate regressions" is just a convenient way to get at all the interactions. Aren't $P(Y_{it}=1|T \geq t)$ and $P(Y_{it}=1|Y_{it-1}=0)$ the same thing? The conditioning in either case says that the unit survived up until time $t$. $\endgroup$ – The Laconic Aug 19 '17 at 14:50
1
$\begingroup$

Approach #1 and approach #2 are the same kind of approach in broad terms. They are both discrete-time hazard models, where the hazard rate takes logistic form, and they are both estimated using logistic regression. The difference is in the choice of independent variables: approach #2 effectively adds interactions between the time factor and every other independent variable. (Although estimated using 36 separate regressions, this is equivalent to estimating it once including full interactions between the time factor as a categorical variable and every other independent variable.)

Since it has (roughly) 36 times more parameters than approach #1, it's not surprising that the AUC and pseudo $R^2$ are much higher. Depending on the number of observations, values close to one could be virtually guaranteed, along with plenty of "insignificant" parameters. Likely, approach #2 is simply overfit, and would not perform well out-of-sample.

Approach #1 might, however, benefit from including interactions between age and other variables. For example, it would not be surprising (based on my own domain knowledge) if default rates varied with age in a way that varied with credit rating.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.