Skip to main content
fix title typo
Link
Ben Reiniger
  • 5.1k
  • 1
  • 12
  • 30

Target encondingencoding in test data and target leakage

Tweeted twitter.com/StackStats/status/1502117218397700148
Became Hot Network Question
Source Link
Chris
  • 535
  • 6
  • 21

Target enconding in test data and target leakage

I understand target encoding, which is the average of the target value by category using out-of-fold mean within each fold. although you get slightly different means for the same value of a categorical variable among different folds. An example (source):

Let's say we have 20-fold cross validation. we need somehow to calculate mean value of the feature for #1 fold using information from #2-#20 folds only.

So, you take #2-#20 folds, create another cross validation set within it. Calculate means for every leave-one-out fold. You average these means and apply that vector for your primary #1 validation set. Repeat that for remaining 19 folds.

My question are:

  1. How using out-of-fold mean within each fold prevents target leakage? Could you give a graphical example or math example or it's too complex with math?
  2. Related with question (1), I have read that regularization prevents the target leakage when using out-of-fold mean within each fold. Which one is it? Or both prevents target leakage?
  3. How do I implement this method in the test set? Because I won't have the target variable, and I can't replace the same value of a categorical variable in the test set with the target mean of the train set with that value of the categorical variable. Why? Because the train set for that value has slightly different means. So, which mean do I put for each specific value in the test set?