Callibration after oversampling

Question

We have build a credit scoring model with OptBinning library in python.

In the process we oversampled the minority class and now we want to callibrate it back to the dataset before oversampling.

This table shows some dummy data of the resulting model, since I cannot show the full model here:

As described in this paper Section 2.2 here, we will run a logistic regression on the dataset before oversampling with just one variable to explain the binary target variable.

This "one variable" can either be the sum of the Points or we could use

$$ {\frac {1}{1+e^{-(\beta _{0}+\beta _{1}x)}}} $$

the probability as calculated on the oversampled dataset. $beta_1 x$ is the sum over WoE * Coefficient for each variable.

What makes more sense in this case and why?

Answer 1

Oversampling the minority class (eg. defaulted customers) will just cause the intercept ($\beta_0$) to artificially increase. You can correct this intercept (estimated based on the oversampled data) as described in section 4.1 of this paper. That said, logistic regression behaves well even when the minority class is only a small fraction of the sample.

If you went with your recalibration approach based on the Points (ie. fitting $\ln(p_i/(1-p_i)) = \beta'_0 + \beta'_1\text{Points}_i$), you'd find that $\beta'_1$ would be close to 1, and $\beta'_0$ would be negative (to adjust the intercept estimated based on the oversampled data).