9
$\begingroup$

Can anybody explain in detail:

  1. What does reject inferencing mean?
  2. How can it be used to increase accuracy of my model?

I do have idea of reject inferencing in credit card application but struggling with the thought of using it to increase the accuracy of my model.

$\endgroup$
22
$\begingroup$

In credit model building, reject inferencing is the process of inferring the performance of credit accounts that were rejected in the application process.

When building an application credit risk model, we want to build a model that has "through-the-door" applicability, i.e., we input all of the application data into the credit risk model, and the model outputs a risk rating or a probability of default. The problem when using regression to build a model from past data is that we know the performance of the account only for past accepted applications. However, we don't know the performance of the rejects, because after applying we sent them back out the door. This can result in selection bias in our model, because if we only use past "accepts" in our model, the model might not perform well on the "through-the-door" population.

There are many ways to deal with reject inferencing, all of them controversial. I'll mention two simple ones here.

  • "Define past rejects as bad"
  • Parceling

"Define past rejects as bad" is simply taking all of the rejected application data, and instead of discarding it when building the model, assign all of them as bad. This method heavily biases the model towards the past accept/reject policy.

"Parceling" is a little bit more sophisticated. It consists of

  1. Build the regression model with the past "accepts"
  2. Apply the model to the past rejects to assign risk ratings to them
  3. Using the expected probability of default for each risk rating, assign the rejected applications to being either good or bad. E.g., if the risk rating has a probability of default of 10%, and there are 100 rejected applications that fall into this risk rating, assign 10 of the rejects to "bad" and 90 of the rejects to "good".
  4. Rebuild the regression model using the accepted applications and now the inferred performance of the rejected applications

There are different ways to do the assignments to good or bad in step 3, and this process can also be applied iteratively.

As stated earlier, the use of reject inferencing is controversial, and it's difficult to give a straightforward answer on how it can be used to increase accuracy of models. I'll simply quote some others on this matter.

Jonathan Crook and John Banasik, Does Reject Inference Really Improve the Performance of Application Scoring Models?

First, even where a very large proportion of applicants are rejected, the scope for improving on a model parameterised only on those accepted appears modest. Where the rejection rate is not so large, that scope appears to be very small indeed.

David Hand, "Direct Inference in Credit Operations", appearing in Handbook of Credit Scoring, 2001

Several methods have been proposed and are used and, while some of them are clearly poor and should never be recommended, there is no unique best method of universal applicability unless extra information is obtained. That is, the best solution is to obtain more information (perhaps by granting loans to some potential rejects) about those applicants who fall in the reject region.

$\endgroup$
  • 1
    $\begingroup$ +1 for extensive overview. Now I know too what reject inferencing is :) $\endgroup$ – mpiktas Jul 28 '11 at 6:51
  • 1
    $\begingroup$ thanks. but how do you assign in step 3 ? I've read that instad of using 1 or 0 you can use the probability for each line. So you will have the same person with 10% and 90%. How can this work with a new logistic model creation? $\endgroup$ – GabyLP Jun 12 '15 at 23:02
1
$\begingroup$

@GabyLP in previous comments. Based on my experience you can split such clients in two parts and assign weights for both of the splits according to probability. For example if a rejected client has 10% PD you can make two clients out of this one. First having target variable 1 and weight 0.1 and second having target variable 0 and weight 0.9.

The whole accepted sample of clients will have weights == 1.

While this works with logistic regression, it does not work with tree based models.

$\endgroup$
  • $\begingroup$ Do you have a source for your statement? $\endgroup$ – T. Beige Nov 29 '18 at 13:52
  • $\begingroup$ If the question is about it not working on tree based models then my answer is - personal experience. I have tried to implement this approach but have not succeeded. $\endgroup$ – MiksL Dec 1 '18 at 9:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.