0
$\begingroup$

In my company I've been noticing some binary classification modeling code that replaces bins of a continuous variable with the corresponding Weight of Evidence (WoE) of the given bin. As far as I understand, WoE for some bin $B_i$ of some variable $X_j$ is calculated by

$$\text{WoE}_{ij} = \ln(\frac{P(X_j \in B_i | Y=1)}{P(X_j \in B_i | Y=0)})$$

where $y=1$ is the positive target and $y=0$ is the negative target of a binary classification problem.

I understand the use case of WoE to help turn a continuous variable to a categorical variable by determining reasonable binning boundaries. What I don't understand is the use case of replacing these bins with the WoE scores themselves.

Doesn't this approach introduce data leakage? Specifically, isn't it problematic that this approach uses $Y$ to transform $X$, which is subsequently used to predict $Y$? In other words, it seems to me that information from $Y$ is indirectly being used to predict itself, but maybe I'm missing something.

This question ostensibly discusses this problem, but OP is mainly concerned with the interpretability of $X$ when taking this kind of approach. I can't find any discussion of data leakage.

$\endgroup$
2
  • 1
    $\begingroup$ After binning, the WoE encoding is just a specialized form of "target encoding". From there, see e.g. stats.stackexchange.com/a/567124/232706 $\endgroup$ Commented Feb 26 at 21:21
  • $\begingroup$ Why do you want to use weight of evidence at all? It is an old idea, better replacements must exist $\endgroup$ Commented Feb 26 at 23:52

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.