3
$\begingroup$

Currently, I am working on a binary classification project that include both numeric and categorical variables as predictors. I recently read an article about encoding variable with weight of evidence in R - http://www.r-bloggers.com/r-credit-scoring-woe-information-value-in-woe-package/. I encoded my categorical variables and also two way interactions of the categorical variables with weight of evidence. I used a 10-fold cross validation and the encoding performed really well on my training set but dismal on my test set. The model was really overfit.

My question is -

  1. When doing such encodings what is the best practice to get the good performance of such transformations while reducing overfitting?

Note: My training set has about 114k records and I am using xgboost as my classification algorithm.

$\endgroup$
2
$\begingroup$

In order to avoid overfitting while using WOE transformations, you should use 'penalty' to discard transformed features that don't hold up against the validation set. Check out the 'information' package: https://cran.r-project.org/web/packages/Information/Information.pdf

Also, here's a more elaborate explanation and demonstration of how 'penalty' should be used: http://multithreaded.stitchfix.com/blog/2015/08/13/weight-of-evidence/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.