1
$\begingroup$

I am attempting to build a model that has many predictors which are both categorical and large in cardinality. Target encoding looks to be a good solution for including these features, but I'm unsure of how to productionize the mapping if k-fold target encoding is used to avoid overfitting as described below:

Article excerpt

Say 5 folds are divided at random; each category in each feature will then have 5 means in the training set. How do I decide which value to map to in production? My only thought is that the folds should be divided deterministically; then in production, we have a mapping for each fold definition - then each event that runs through our production model will have its fold calculated in realtime and mapped according to that value?

Image taken from: https://towardsdatascience.com/getting-deeper-into-categorical-encodings-for-machine-learning-2312acd347c8

$\endgroup$
2
  • 2
    $\begingroup$ This is more than a "productionization" problem. How do you even evaluate this model on a test set? It's an interesting technique though. Edit: their Jupyter notebook doesn't even define all the functions they use in the code, kind of a mess. $\endgroup$ Jun 23 '20 at 16:40
  • 1
    $\begingroup$ Great point - I believe the answer to this question and yours could be one and the same. $\endgroup$
    – Darrrrrren
    Jun 23 '20 at 16:41
2
$\begingroup$

The answer is in the source code for the blog post: https://github.com/groverpr/Machine-Learning/blob/9963e59823fe0ff18cc8e1b2657b71c01f133193/catboost/utils.py#L25-L27

They re-fit the target encoding using the entire training set, before making out-of-sample predictions.

Note that they also fill missing values with the global mean of the target.

Edit: I wonder if it could be useful to average together the results of the K encodings, rather than re-training a single encoding on the entire dataset.

$\endgroup$
2
  • $\begingroup$ To clarify, when you say they re-fit the target encoding using the entire training set, you mean they essentially re-do the means calculation, just without folds? $\endgroup$
    – Darrrrrren
    Jun 23 '20 at 17:17
  • 1
    $\begingroup$ Yes @Darrrrrren $\endgroup$ Jun 23 '20 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.