0
$\begingroup$

I started learning about survival analysis only recently, so I am not sure if this question fits in survival analysis. If you think I lack the background to do what I am trying to do, please point me to good references.


The problem I am trying to solve is in the lending domain. There are various loan products that started about 18 months back. All loans have an 180 month term. The loan statuses can be grouped into three cases:

  1. About 5% of the loans have already been paid off.
  2. About 13% went delinquent (for 30 days) at least once.
  3. Rest of the loans (about 80%) never went delinquent and are not paid off.

As you can see, this is a heavily "right censored" data set, since there are a lot of loans in the third category.

We have about 700 features of our borrowers (mostly demographic features) and have issued about 5000 loans so far. The goal is to find the borrower groups that are performing well and channel our advertising dollars to those groups i.e. ad targeting.

One approach to solve the problem is to look at loans that have some history, say 6 months, and do a good/bad split. If I take only the charged off loans as bad, then I have only 5% bad examples, leading to a heavily skewed dataset. I am thinking of any loan that went delinquent as bad (even if it caught up later). I think loans that have been paid off can also be considered bad, since they didn't make enough interest income. Most likely these two groups of people are different from the people who are current on their loans, so a classifier can distinguish the good and bad. This gives about 80/20 split between good and bad cases. Then I can do classification and look at feature importances to figure out who the good borrowers are.

The main problem I see with this classification approach is that I get only 70% of the data points. Also most "paid off" loans are paid off within a month. This is why I started looking at survival analysis, thinking it will help solve the censoring problem.

I read Crawley's "The R Book" chapter on survival analysis, but I am not sure how to identify distinguishing features between the groups. The examples show survival plots for categorical features with few levels and two features at max, with an interaction term included. I am at a loss as to how to apply that approach to my problem.

Does my problem fit into survival analysis? If yes, how can I use it on a high dimensional dataset like the one I have and figure out feature importances? Or is my classification approach good enough?

(Any other approaches to solve this problem are also welcome.)

$\endgroup$
  • 1
    $\begingroup$ You can certainly use survival analysis see eg peerlendingserver.com/uncategorized/1310. The problem is that you have only 5000 loans. This means that even measuring the overall default rate has a large margin of error. (do simulation with known default rate and see the variability you get in observed default rate). So basically all you can hope for is to estimate the coefficient for the credit score variable regardless of classification or survival analysis. $\endgroup$ – seanv507 Dec 8 '16 at 5:33
  • $\begingroup$ Thank you very much for referring the blog! I see many relevant posts there. $\endgroup$ – arun Dec 8 '16 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.