I am aware of the fact that categorical variables with k levels should be encoded with k-1 variables in dummy encoding (similarly for multi-valued categorical variables). I was wondering how much of a problem does a one-hot encoding (i.e. using k variables instead) over dummy encoding for different regression methods, mainly linear regression, penalized linear regression (Lasso, Ridge, ElasticNet), tree-based (random forests, gradient boosting machines).

I know that in linear regression, multi-collinearity problems occur (even though in practice I have fitted linear regression using OHE without any issues).

However, does dummy encoding need to be used in all of them and how wrong would the results be if one uses one-hot encoding?

My focus is on prediction in regression models with multiple (high-cardinality) categorical variables, so I am not interested in confidence intervals.

  • 10
    $\begingroup$ I claim it is incorrect to distinguish the two. Dummy variables and one-hot variables are complete synonyms. The first term is older and comes from statistics, while the second is younger and comes from machine learning. The third, and more formal synonym is indicator type contrast variables. The question whether to use all k or k-1 nonredundant variables in the set of such elementary variables has nothing to do with that terminology and depends on the type of analysis and on the concrete algorithm or program. $\endgroup$ – ttnphns Jul 9 '17 at 21:22
  • 2
    $\begingroup$ (cont.) For example, a regression software will typically not allow to enter all k because of singularity, but equivalent general linear modeling software may allow it since it uses pseudoinverse approach. $\endgroup$ – ttnphns Jul 9 '17 at 21:22
  • 1
    $\begingroup$ I'm with @ttnphns, both of those are just awful names. I prefer full-encoding and leave-one-out encoding. $\endgroup$ – Matthew Drury Jul 10 '17 at 20:38
  • 2
    $\begingroup$ Apologies for the terminology, this is the standard terms I have seen used by practitioners (and similarly oriented books). The issue of collinearity appears only in linear (unpenalized models)? Not all software stops from entering all k (e.g. Python's scikit-learn, correct me if I am wrong) $\endgroup$ – user90772 Jul 11 '17 at 23:46
  • $\begingroup$ @MatthewDrury One of my pet peeves are duplicate terms/phrases creeping into the field of statistical analysis, like "one-hot encoding" and "A/B testing". Everyone should stick with the older "dummy coding" and "hypothesis testing" to avoid confusion. $\endgroup$ – RobertF Feb 4 '19 at 20:27

The issue with representing a categorical variable that has $k$ levels with $k$ variables in regression is that, if the model also has a constant term, then the terms will be linearly dependent and hence the model will be unidentifiable. For example, if the model is $μ = a_0 + a_1X_1 + a_2X_2$ and $X_2 = 1 - X_1$, then any choice $(β_0, β_1, β_2)$ of the parameter vector is indistinguishable from $(β_0 + β_2,\; β_1 - β_2,\; 0)$. So although software may be willing to give you estimates for these parameters, they aren't uniquely determined and hence probably won't be very useful.

Penalization will make the model identifiable, but redundant coding will still affect the parameter values in weird ways, given the above.

The effect of a redundant coding on a decision tree (or ensemble of trees) will likely be to overweight the feature in question relative to others, since it's represented with an extra redundant variable and therefore will be chosen more often than it otherwise would be for splits.

  • 1
    $\begingroup$ I don't think there is any reason to include an intercept in a tree or ensemble of trees. I believe that is a thing particular to linear type models. So for tree based models, I think the right idea is not intercept, full encoding. $\endgroup$ – Matthew Drury Jul 10 '17 at 20:36
  • $\begingroup$ @MatthewDrury I think you're right about the intercept, but even then, for a tree, redundant coding seems of limited use. For example, if a feature is binary, what's the difference between splitting on the first class and splitting on the second class? Nothing, so far as I can tell. $\endgroup$ – Kodiologist Jul 10 '17 at 20:44
  • $\begingroup$ It's true, but I think keeping the full symmetry is the simplest way to keep everything obviously fair and interpretable. $\endgroup$ – Matthew Drury Jul 10 '17 at 20:50

I feel the best answer to this question is buried in the comments by @MatthewDrury, which states that there is a difference and that you should use the seemingly redundant column in any regularized approach. @MatthewDrury's reasoning is

[In regularized regression], the intercept is not penalized, so if you are inferring the effect of a level as not part of the intercept, its hard to say you are penalizing all levels equally. Instead, always include all the levels, so each is symmetric with respect to the penalty.

I think he's got a point.

  • $\begingroup$ So you should one hot encode variable with k levels or with k-1 levels depending on the situation. In addition to your statement (regularized/not regularized), would there be guidelines on what to do in all cases ? $\endgroup$ – Dan Chaltiel Mar 20 '19 at 8:22
  • $\begingroup$ My rule is if there's no regularization, as in classical ANOVA, use k-1 levels. If there is regularization, as in Bayesian methods or regression with L2 regularization, use k levels. $\endgroup$ – Ben Ogorek Dec 29 '19 at 14:32

Kodiologist had a great answer (+1). One-hot encoding vs. dummy encoding encoding methods are the same, in terms of the design matrix are in the same space, with different basis. (although the one-hot encoding has more columns)

Therefore if you are focusing on accuracy instead of interpretability. Two encoding methods makes no difference.

  • 1
    $\begingroup$ To be pedantic, the one-hot version isn't a basis (because it's linearly dependent); it just spans the same space. But are you sure encoding makes no difference for accuracy? Particularly in the case of penalized regression, I think the final selected model will make different predictions. $\endgroup$ – Kodiologist Jul 9 '17 at 2:37
  • $\begingroup$ @Kodiologist thanks for correct me on the basis issue. $\endgroup$ – Haitao Du Jul 9 '17 at 2:40
  • $\begingroup$ @Kodiologist why the accuracy will be different in regularized linear method? $\endgroup$ – Haitao Du Jul 9 '17 at 2:40
  • 6
    $\begingroup$ In regularized regression, you should always use a full encoding (i.e. not leave-one-out, I can't keep the names you used straight, and prefer not to use them myself). This is because the intercept is not penalized, so if you are inferring the effect of a level as not part of the intercept, its hard to say you are penalizing all levels equally. Instead, always include all the levels, so each is symmetric with respect to the penalty. $\endgroup$ – Matthew Drury Jul 10 '17 at 20:35
  • 1
    $\begingroup$ @Matthew Drury, thank you for this last comment. Could you please expand it in an answer? So it is only for plain linear regression where dummy encoding is necessary? $\endgroup$ – user90772 Jul 11 '17 at 23:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.