I'm working on a credit binary classification task.For this business is something usual to meassure model's performance from two metrics: ROC AUC and KS .This sounds reasonable until I have to choose between two models with "contradictory" results. Let's say I have the following:
Model 1 : AUC: .90 & KS: .70
Model 2 : AUC: .85 & KS: .80
I have been thinking on a way to combine these two (or even more metrics) to make a final decision on which model is better.
1. What would be a correct way of combining multiple metrics to make a decision of a model to be selected?
2. What are the characteristics this combination should fulfill? (limit to infinity 1 , limit to minus infinity 0)
I'm looking for either a formula to combine multiple metrics into one score to be able to make decisions based on that or a methodology to take into consideration more than one metric to make decisions