Is one-hot encoding required for a binary categorical variable?

Question

We are performing multiple linear regression.

Dataset:

• let's call the response Y and the predictors X1, X2 and X3,
• where Y, X1, X2 are continuous, whereas X3 is binary categorical variable (i.e. it can take either two values, 1 or 2).

Now, we have two choices

1. Choice #1: Without one-hot encoding of X3

   • i.e. Linear regression on raw variables lm(Y ~ X1 + X2 + X3)

2. Choice #2: With one-hot encoding of X3

   • i.e. X3 is one-hot encoding in [X3a, X3b]
   • e.g. Possible values: [1, 0] and [0, 1]
   • Hence, the linear regression model is lm(Y ~ X1 + X2 + X3a + X3b)

Question:

Since X3 is a binary variable, whether or not we one-hot encode (i.e. Choice#1 or Choice#2), will it make any difference (especially in terms of the model's loss)?

Answer 1

Both models will yield exactly the same predictions -- but they might be harder to interpret than standard encodings. Let's look at them.

Please note, though, that by "binary categorical variable ... 1 or 2" we must understand that the values of X3 are treated by lm as literally being the numbers $1$ or $2.$ If X3 is a factor variable (which is how one ordinarily would store a categorical value), then lm will internally convert one of the values to $1$ and the other to $0$.

First model

The formula string ~ X1 + X2 + X3 implicitly includes an intercept. Mathematically -- and more explicitly -- the model is

$$E[Y] = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_3$$

where the $X_i$ are numerical random variables and the $\beta_i$ are the coefficients to be estimated. This gives two expressions corresponding to the two values of $X_3:$

1. When $X_3=1,$ the model is $E[Y] = (\beta_0 + \beta_3) + \beta_1X_1 + \beta_2 X_2.$

2. When $X_3=2,$ the model is $E[Y] = (\beta_0 + \beta_3) + \beta_1 X_1 + \beta_2 X_2 + \beta_3.$

I wrote these expressions in a somewhat strange way to make it clear that this model is equivalent to $$E[Y] = \alpha_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 \mathscr{I}(X_3=2)$$ in which $\mathscr{I}(X_3=2)$ is the "one-hot encoding" for $X_3$ and, evidently, $\alpha_0 = \beta_0 + \beta_3.$

In this sense the first model is equivalent to what lm would do with a categorical variable, but the intercept it reports (namely, $\beta_0$) will differ (in a predictable way) from the usual intercept.

Second model

I will continue to use the standard indicator notation, so that X3a is mathematically expressed as $\mathscr{I}(X_3=1)$ and X3b is $\mathscr{I}(X_3=2).$ This model therefore is

$$\begin{aligned} E[Y] &= \gamma_0 + \gamma_1X_1 + \gamma_2 X_2 + \gamma_3 \mathscr{I}(X_3=1) + \gamma_4 \mathscr{I}(X_3=2)\\ &= (\gamma_0 - \delta) + \gamma_1 X_1 + \gamma_2 X_2 + (\gamma_3 + \delta)\mathscr{I}(X_3=1) + (\gamma_4 + \delta)\mathscr{I}(X_3=2). \end{aligned}$$

The second line comes from adding and subtracting $\delta$ from the right hand side and using

$$1 = \mathscr{I}(X_3=1) + \mathscr{I}(X_3=2).$$

That is, since X3 is either 1 or 2, the sum of the two "one-hot" codes is constantly $1.$

The point is that including all three of an intercept, X3a, and X3b is redundant. The statistical term for this situation is that the model is not identifiable. lm will handle this by recognizing the redundancy by the time it processes the + X3b term and will just drop that term, reporting its coefficient as NA. Consequently, the lm implementation is

$$E[Y] = \gamma_0 +\gamma_1X_1 + \gamma_2 X_2 + \gamma_3 \mathscr{I}(X_3=1).$$

As in the first model, there are two cases for the two possible values of $X_3:$

1. When $X_3=1,$ the model is $E[Y] = (\gamma_0 + \gamma_3) + \gamma_1X_1 + \gamma_2 X_2 + \gamma_3.$

2. When $X_3=2,$ the model is $ (\gamma_0 + \gamma_3) + \gamma_1X_1 + \gamma_2 X_2 - \gamma_3.$

Again I have written these to aid in the comparison with the first model. Evidently $\gamma_0 + \gamma_3 = \alpha_0$ and $\gamma_3 = -\beta_3.$

In this sense the second model is equivalent to what lm would do with a categorical variable, but it will report a (predictably) different intercept and the coefficient of the term for $X_3$ will be negated.

---

Having done the analysis, let's compare with what lm does. I created a tiny data frame (just four observations), but it's enough to yield estimates. In this model $\alpha_0 = 1,$ $\beta_1 = -2,$ $\beta_2 = -1,$ and $\beta_3 = 3.$

X <- data.frame(x1 = 1:4,
                x2 = (1:4)^2,
                x3 = rep(1:2, each=2))

b <- c(1,-2,-1,3)
X$y <- with(X, b[1] + b[2] * x1 + b[3] * x2 + b[4] * ifelse(x3 == 2, 1, 0))

# Default `lm` result
(lm(y ~ x1 + x2 + factor(x3), X))

# First model
(lm(y ~ x1 + x2 + x3, X))

# Second model
X$x3a <- ifelse(X$x3 == 1, 1, 0)
X$x3b <- ifelse(X$x3 == 2, 1, 0)
(lm(y ~ x1 + x2 + x3a + x3b, X))

The output for the three models lists the estimated coefficients:

lm(formula = y ~ x1 + x2 + factor(x3), data = X)
(Intercept)           x1           x2  factor(x3)2  
          1           -2           -1            3  

lm(formula = y ~ x1 + x2 + x3, data = X)
(Intercept)           x1           x2           x3  
         -2           -2           -1            3  

The estimate of $-2$ for the intercept is the difference $1-3$ (intercept minus x3 coefficient) from the first output.

lm(formula = y ~ x1 + x2 + x3a + x3b, data = X)
(Intercept)           x1           x2          x3a          x3b  
          4           -2           -1           -3           NA  

The estimate of $4$ for the intercept is the sum $1+3$ (intercept plus x3 coefficient) from the first output; the estimate of $-3$ for the x3a term is the negative of the x3 coefficient from the first output; and NA is reported for the x3b term.

All three models (lm, first, and second) are equivalent, but the meanings of their intercepts and coefficients related to x3 vary.

Answer 2

There's a better approach than both. With dummy coding, you would end up with a simpler model that is easier to interpret.

In the first case, you are forcing $\beta_3 X_3$ to be twice higher (or lower) for group coded 2 vs 1. To adapt to a specific difference between groups, the parameter, and intercept need to adapt. It's much easier to interpret the parameters using 0 and 1 codes, so $\beta_3 \times 1$ (vs $0$) serves as an additional intercept for the group coded as 1.

In the second case $X_{3a}$ and $X_{3b}$ are colinear and cannot be used. What you need to do is to encode the categories as 0 and 1, so you encode the presence of one category that has an additive effect $\beta_3 X_3 = \beta_3 1 = \beta_3$.

Answer 3

@Tim,

I think both would give same performance because:

• Rationale#1:Mathematically, there's enough room for parameter to adjust (to give same y value).
• Rationale#2: Computationally, we get the same explanatory power (i.e. R^2 value). Minimal reproducible example:

# Read Data
data("mtcars")
data <- mtcars
attach(data)

# Model 1: My Model
am_new <- am+1
model_1_my <- lm(mpg ~ am_new)
print(summary(model_1_my))        # Output: R-squared:  0.3598

# Model 2: Tim's Model
model_2_tim <- lm(mpg ~ am)
print(summary(model_2_tim))       # Output: R-squared:  0.3598   (again)