Suppose we have a machine-learning model already in use in the context of credit card application approval process: the application will be successful if the model predicts that the applicant is a good customer, and the application will fail if the applicant is predicted to be a bad customer or even a fraud.
The model has been running for several years, and has a decent performance (within the customers who own our credit card, only 5% of them are bad customers). now it is time to improve our existing models, the question is how?
we have all the data of all the applicants regardless of their applications being successful or not. following are the several options plus questions:
we consider all the failed customers whose application didn't pass through as bad customers, and we consider those 5% bad customers as, obviously, bad customers. Then I train a binary classification model. the model might gives us a lower percentage of bad customers (lower false negative number), with the price of much higher false positive number, which means less people will be accepted as potential customer, and the model might turn out to be completely useless. how to fix that?
the ideal model will gave us higher acceptation rate and lower bad customer rate, which is almost impossible from a model. the second would be higher acceptation rate and a slightly higher bad customer rate, how to use our data to reach this goal?
are there other methods to improve the existing model?