2
$\begingroup$

My team has been implementing a survival-analysis approach to loan default prediction for about a year now. However, I believe their method to be a little "brute-force" and I haven't been able to compare it to anything I've read online about survival models.

A typical loan dataset would have one row per application and an outcome flag indicating if the customer defaulted within the first 12 months after application. It is a simple binary classification task that eventually aims to construct the probability of default (or PD) at 12 months.

Survival models have some obvious advantages in this use case. The two biggest are that you can use applications that haven't been observed for the full 12 months (or is right-censored) and that there is now a time component which is useful for more detailed analysis and model monitoring.

The way that my team has the dataset set up for "survival modelling" is like-so (with a credit score shown to represent the features we have available at the time of application):

Customer Credit score Months since application Late
A 240 1 0
A 240 2 0
A 240 3 1
B 270 1 0
B 270 2 0
B 270 3 0
B 270 4 1
C 400 1 0
C 400 2 0

As you can see, we duplicate the information available at the time of application (the credit score) by the number of months since application and include this time as another column in the dataset. The late flag is included only on the month that the customer actually defaults and no further months are included for this customer. We then build a gradient-boosting classification model directly from this dataset, with no formal survival analysis package or equations being used. This means that we essentially view time in the same way as any other factor in the model, and can plot its effect on the outcome flag in the same way.

I have doubts about this approach based mostly on gut feel and the fact I've not seen it done in this way anywhere in my research. Intuition tells me this approach is inefficient and doesn't properly consider the effects of censoring, but unfortunately, I am lacking the deep statistical understanding of survival modelling required back this up.

Can anyone see the faults with this approach, if any? What are the disadvantages of doing it this way, and how should I correctly implement a gradient-boosting model for survival analysis?

$\endgroup$
1
  • $\begingroup$ The "duplicating information" is really just reflecting the 'repeated measures'/'hierarchical' nature of your data structure (times of measurement nested within individuals). $\endgroup$
    – Alexis
    Commented May 16, 2023 at 14:05

1 Answer 1

3
$\begingroup$

For each time period, you show one row of data for each individual at risk of the event during that period, with covariate values in place for the individual during that period and an indicator of whether the event occurred. That's a standard "long form" or "person-period" data format used for what's called "discrete-time survival analysis." That allows you to convert survival analysis into a binomial regression when you have a limited number of time periods, as in your case.

Right censoring of event times is handled just as it is in continuous time: an individual's covariate values are included in the analysis while the individual is at risk, but aren't included thereafter. You are free to model the time aspect however you wish, as categorical or with an underlying continuous function.

From that perspective, using gradient-boosting instead of a standard binomial regression is as appropriate here as it is for any binary outcome. It's best to use a continuous measure rather than yes/no classification accuracy to guide the model, but otherwise there's no inherent problem.

This page, among others on this site, outlines the approach and includes some links to references. There's a 2016 text by Tutz and Schmid, Modeling Discrete Time-to-Event Data, that goes into much detail, although the reading is rough in some places.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. Could you explain a bit more about your comment "It's best to use a continuous measure rather than yes/no classification accuracy to guide the model"? Are you talking about the objective function here? I'm using CatBoostClassifier with this data structure, with time being a feature of the model as you describe, and the outcome is a binary late flag. The standard objective function is LogLoss, and the others available all look like classification objectives to me. Should I be using CatBoostRegressor, even with the binary outcome? $\endgroup$
    – jgk_iles
    Commented May 18, 2023 at 10:59
  • 2
    $\begingroup$ @jgk_iles yes, I'm talking about the objective function. See this page and its links for extensive discussion about why accuracy of classification is not a good choice. Log loss, Brier score, other strictly proper scoring rules tend to provide better calibrated probabilities of outcomes, which you can combine with relative mis-classification financial costs to choose a probability cutoff that minimizes net cost. Accuracy typically invokes a cutoff at 50% probability, not always the best choice. $\endgroup$
    – EdM
    Commented May 18, 2023 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.