# Why should one do a WOE transformation of categorical predictors in logistic regression?

When is weight of evidence (WOE) transformation of categorical variables useful?

The example can be seen in WOE transformation

(So for a response $y$, & a categorical predictor with $k$ categories, & $y_j$ successes out of $n_j$ trials within the $j$th category of this predictor, the WOE for the $j$th category is defined as

$$\log \frac{y_j} {\sum_j^k {y_j}} \frac{\sum_j^k (n_j-y_j)}{n_j-y_j}$$

& the transformation consists of coding each category of the categorical predictor with its WOE to form a new continuous predictor.)

I would like to learn the reason why WOE transformation helps the logistic regression. What is the theory behind this?

In the example you link to, the categorical predictor is represented by a single continuous variable taking a value for each level equal to the observed log odds of the response in that level (plus a constant):

$$\log \frac{y_j} {n_j-y_j} + \log \frac{\sum_j^k (n_j-y_j)}{\sum_j^k {y_j}}$$

This obfuscation doesn't serve any purpose at all that I can think of: you'll get the same predicted response as if you'd used the usual dummy coding; but the degrees of freedom are wrong, invalidating several useful forms of inference about the model.

In multiple regression, with several categorical predictors to transform, I suppose you'd calculate WOEs for each using marginal log odds. That will change the predicted responses; but as confounding isn't taken into account—the conditional log odds aren't a linear function of the marginal log odds—I can't see any reason to suppose it an improvement, & the inferential problems remain.

• Can you explain why degrees of freedom is wrong with WOE? It is just a transformation right? Also what if we had several categorical variables, and we got WOE for each of the one by one? In my experience when you have many categorical variables, then some buckets between different variables overlap a lot, and you start to see some coefficients which are insignificant. And also you need to carry around several coefficients.
Commented Aug 12, 2015 at 13:35
• (1) A transformation that depends on evaluating the relation of predictors to response - something that's supposed to be left to the regression. So e.g. the likelihood ratio test statistic won't have the same distribution as when a transformation is pre-specified. (2) Good point! - a multiple regression on WOEs won't be equivalent to that on dummy variables (unless the models are saturated). (3) So what? (4) Coefficients aren't any heavier than WOEs. Commented Aug 12, 2015 at 14:56
• I guess WoE is a leftover from times where computation where more of a problem than today. So maybe, with categorical predictors with MANY levels, converting to a numeric variable was a bright idea! Commented Apr 23, 2020 at 4:25

Coarse classing using the measure of weight of Evidence (WoE) has the following advantage- WoE displays a linear relationship with the natural logarithm of the odds ratio which is the dependent variable in logistic regression.
Therefore, the question of model misspecification does not arise in logistic regression when we use WoE instead of the actual values of the variable.

$ln(p/1-p)$ = $\alpha$ + $\beta$*$WoE(Var1)$ + $\gamma$*$WoE(Var2)$ + $\eta$*$WoE(Var3 )$

Source: In one of the PPTs my trainer showed me during the company training.

• "model misspecification does not arise in logistic regression when we use WoE instead of the actual values of the variable". Can you explain/prove this mathematically?