I am currently developing a loan default risk model using a discrete time hazard approach with xgboost. The goal is to generate a series of predicted monthly default probabilities using a new borrowers features at the time of origination.

The training data set contains monthly snapshots of each loan's payment history until the loan terminates via payoff or default. The data is set up as a binary classification problem with the target variable indicating if the borrower paid or defaulted that month. This indicator, plus the variable for loan age are the only time varying features in this data set. The rest are repeating features collected at the time of origination such as credit info and loan characteristics such as term length and loan amount.

The problem here is that some of the loans are right censored, so the loan history is not available for later months, especially for those that originated in recent years. What are some solutions to this problem? Should I remove the censored data?


1 Answer 1


Should I remove the censored data?

That depends on what you mean by "remove."

You need to include all cases at times when they are at risk of an event. Otherwise you risk substantial bias. So you do not remove all data from individuals who end up having censored survival times.

If an individual is not at risk after a particular time, however, then that individual should not be included in the analysis at those later times. Anything that erroneously includes that individual in a risk set after that time (e.g., a data row for an observation time at which the individual could not have been observed) should be removed. That's standard practice for right censoring.

  • $\begingroup$ My apologies, I meant remove the loans from the training data set. $\endgroup$
    – tatakae888
    Jun 23, 2022 at 19:55
  • $\begingroup$ @user8210610 so you shouldn't do that. Structure the data and analysis so that those loans aren't used in calculations at "survival" times when they are no longer under observation. $\endgroup$
    – EdM
    Jun 23, 2022 at 20:04
  • $\begingroup$ With the exception of right censored loans, the data is structured so that the monthly observations end once the loan defaults or pays off. So, the data only includes at-risk loan/month records. However, the data includes loans originated between 2017 to 2018 with monthly payment history up to 2021. As a result, there are loans with 60 and 72 month term lengths that are right censored. How would this affect the model's predicted monthly default probabilities for loans with 60 and 72 month term lengths given that the model has never seen examples of records in these later periods? $\endgroup$
    – tatakae888
    Jun 24, 2022 at 0:48
  • $\begingroup$ @user8210610 the model will use the available information about loans of such terms, but it can't provide any information about such loans' probability of default beyond the duration of observation. As I understand your description, that means you won't have any information about defaults on 72-month loans beyond 60 months into the term (earliest origination, January 2017; latest payment information, December 2021). The limiting factor is the number of observed "events" (defaults) after any "survival time"; the fewer, the less precise the estimate. $\endgroup$
    – EdM
    Jun 24, 2022 at 2:04
  • $\begingroup$ Thanks @EdM much appreciated. Your understanding is correct. However, there are varying origination dates and varying term lengths which result in a data set where 30% of the loans are right censored. This percentage is much higher when looking at loans originated in 2018. $\endgroup$
    – tatakae888
    Jul 1, 2022 at 23:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.