I have a task to create credit scoring model using WOE encoding. I have a very small dataset, so I wont be able to perform testing on test and out-of-time samples. Thus, I am going to use cross-validation.
Can you please tell me which variant is correct:
1 Variant. I perform binning on my entire dataset. I calculate WOE. I replace the values of my variables by their WOE encoding. Then start cross-validation on 3 folds. I just split my entire dataset with WOE values on 3 parts and do cv. This leads to bias. In every iteration of cv, I have train and test samples, but WOE values in test samples were calculated using binning applied to entire dataset so WOE values in test sample are influenced by test sample. Does it sound correct? Is there a data leakage?
2 Variant. First I split my dataset according to 3 fold cv. Afterwards each iteration is processed in a such way that I perform binning and calculate WOEs on train sample, and then I replace values in test sample with those WOEs. 3 iterations = 3 different binnings. But WOEs from test sample will not be influenced by test sample because binning was performed on each iteration using only train dataset.
3 Variant. First I split my dataset according to 3 fold cv. Afterwards each iteration is processed in a such way that I use binning performed on entire dataset but calculate WOEs based on train sample, and then I replace values in test sample with those WOEs. 3 iterations = 3 different WOEs with same binning. Is there a data leakage?
Can you please tell me which way is correct and what do you think? I would be extremely grateful if you could share any publications and researches on cross-validation with WOE transformation.
Thanks!
