2
$\begingroup$

In the H2O tutorial on target encoding they recommend fitting a new TargetEncoder to the entire training set to encode the test data. Why not just use the averaged TargetEncoder fit to the training data in the k-fold cross validation?

I understand their point about overfitting not being a risk, but it seems to me like they are introducing a change in the distribution of the encoded variable between the training and test sets.

http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-munging/target-encoding.html#transform-target-encoding

$\endgroup$
0
$\begingroup$

The point the documentation is trying (perhaps not clearly) to make is that cross-validation can/should be used to evaluate target-encoding a high-cardinal variable. If the target-encoded variable model shows a small CV error, you should then use target-encoding. We can then use all of the data when target-encoding a high cardinal variable. The main point: ** models created during cross validation are used to assess your model-building decisions, e.g. target encoding; they are NOT the final models you use in practice- You use your entire training set.**

We cannot assess the final target-encoding variable with the training error. We will need another test set to get a better idea of the generalization error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.