# Is one-hot encoding required for a binary categorical variable?

We are performing multiple linear regression.

Dataset:

• let's call the response Y and the predictors X1, X2 and X3,
• where Y, X1, X2 are continuous, whereas X3 is binary categorical variable (i.e. it can take either two values, 1 or 2).

Now, we have two choices

1. Choice #1: Without one-hot encoding of X3

• i.e. Linear regression on raw variables lm(Y ~ X1 + X2 + X3)
2. Choice #2: With one-hot encoding of X3

• i.e. X3 is one-hot encoding in [X3a, X3b]
• e.g. Possible values: [1, 0] and [0, 1]
• Hence, the linear regression model is lm(Y ~ X1 + X2 + X3a + X3b)

Question:

Since X3 is a binary variable, whether or not we one-hot encode (i.e. Choice#1 or Choice#2), will it make any difference (especially in terms of the model's loss)?

• In my opinion - Though model parameters: β's will be different - but the final loss (goodness of fit) should be same, isn't it?
– pqrz
Jul 11, 2022 at 6:04
• Yes, that's correct. There are subtleties, though, both statistical and computational. You need to understand what lm is doing in order to interpret its output. The default tests that lm performs will be slightly different, too, because the meanings of some of the parameters vary from one model to the next. Finally, there are far more than two ways to write an equivalent model!
– whuber
Jul 16, 2022 at 16:29

Both models will yield exactly the same predictions -- but they might be harder to interpret than standard encodings. Let's look at them.

Please note, though, that by "binary categorical variable ... 1 or 2" we must understand that the values of X3 are treated by lm as literally being the numbers $$1$$ or $$2.$$ If X3 is a factor variable (which is how one ordinarily would store a categorical value), then lm will internally convert one of the values to $$1$$ and the other to $$0$$.

### First model

The formula string ~ X1 + X2 + X3 implicitly includes an intercept. Mathematically -- and more explicitly -- the model is

$$E[Y] = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_3$$

where the $$X_i$$ are numerical random variables and the $$\beta_i$$ are the coefficients to be estimated. This gives two expressions corresponding to the two values of $$X_3:$$

1. When $$X_3=1,$$ the model is $$E[Y] = (\beta_0 + \beta_3) + \beta_1X_1 + \beta_2 X_2.$$

2. When $$X_3=2,$$ the model is $$E[Y] = (\beta_0 + \beta_3) + \beta_1 X_1 + \beta_2 X_2 + \beta_3.$$

I wrote these expressions in a somewhat strange way to make it clear that this model is equivalent to $$E[Y] = \alpha_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 \mathscr{I}(X_3=2)$$ in which $$\mathscr{I}(X_3=2)$$ is the "one-hot encoding" for $$X_3$$ and, evidently, $$\alpha_0 = \beta_0 + \beta_3.$$

In this sense the first model is equivalent to what lm would do with a categorical variable, but the intercept it reports (namely, $$\beta_0$$) will differ (in a predictable way) from the usual intercept.

### Second model

I will continue to use the standard indicator notation, so that X3a is mathematically expressed as $$\mathscr{I}(X_3=1)$$ and X3b is $$\mathscr{I}(X_3=2).$$ This model therefore is

\begin{aligned} E[Y] &= \gamma_0 + \gamma_1X_1 + \gamma_2 X_2 + \gamma_3 \mathscr{I}(X_3=1) + \gamma_4 \mathscr{I}(X_3=2)\\ &= (\gamma_0 - \delta) + \gamma_1 X_1 + \gamma_2 X_2 + (\gamma_3 + \delta)\mathscr{I}(X_3=1) + (\gamma_4 + \delta)\mathscr{I}(X_3=2). \end{aligned}

The second line comes from adding and subtracting $$\delta$$ from the right hand side and using

$$1 = \mathscr{I}(X_3=1) + \mathscr{I}(X_3=2).$$

That is, since X3 is either 1 or 2, the sum of the two "one-hot" codes is constantly $$1.$$

The point is that including all three of an intercept, X3a, and X3b is redundant. The statistical term for this situation is that the model is not identifiable. lm will handle this by recognizing the redundancy by the time it processes the + X3b term and will just drop that term, reporting its coefficient as NA. Consequently, the lm implementation is

$$E[Y] = \gamma_0 +\gamma_1X_1 + \gamma_2 X_2 + \gamma_3 \mathscr{I}(X_3=1).$$

As in the first model, there are two cases for the two possible values of $$X_3:$$

1. When $$X_3=1,$$ the model is $$E[Y] = (\gamma_0 + \gamma_3) + \gamma_1X_1 + \gamma_2 X_2 + \gamma_3.$$

2. When $$X_3=2,$$ the model is $$(\gamma_0 + \gamma_3) + \gamma_1X_1 + \gamma_2 X_2 - \gamma_3.$$

Again I have written these to aid in the comparison with the first model. Evidently $$\gamma_0 + \gamma_3 = \alpha_0$$ and $$\gamma_3 = -\beta_3.$$

In this sense the second model is equivalent to what lm would do with a categorical variable, but it will report a (predictably) different intercept and the coefficient of the term for $$X_3$$ will be negated.

Having done the analysis, let's compare with what lm does. I created a tiny data frame (just four observations), but it's enough to yield estimates. In this model $$\alpha_0 = 1,$$ $$\beta_1 = -2,$$ $$\beta_2 = -1,$$ and $$\beta_3 = 3.$$

X <- data.frame(x1 = 1:4,
x2 = (1:4)^2,
x3 = rep(1:2, each=2))

b <- c(1,-2,-1,3)
X$y <- with(X, b + b * x1 + b * x2 + b * ifelse(x3 == 2, 1, 0)) # Default lm result (lm(y ~ x1 + x2 + factor(x3), X)) # First model (lm(y ~ x1 + x2 + x3, X)) # Second model X$$x3a <- ifelse(X$$x3 == 1, 1, 0) X$$x3b <- ifelse(X$$x3 == 2, 1, 0) (lm(y ~ x1 + x2 + x3a + x3b, X))  The output for the three models lists the estimated coefficients: lm(formula = y ~ x1 + x2 + factor(x3), data = X) (Intercept) x1 x2 factor(x3)2 1 -2 -1 3 lm(formula = y ~ x1 + x2 + x3, data = X) (Intercept) x1 x2 x3 -2 -2 -1 3  The estimate of $$-2$$ for the intercept is the difference $$1-3$$ (intercept minus x3 coefficient) from the first output. lm(formula = y ~ x1 + x2 + x3a + x3b, data = X) (Intercept) x1 x2 x3a x3b 4 -2 -1 -3 NA  The estimate of $$4$$ for the intercept is the sum $$1+3$$ (intercept plus x3 coefficient) from the first output; the estimate of $$-3$$ for the x3a term is the negative of the x3 coefficient from the first output; and NA is reported for the x3b term. All three models (lm, first, and second) are equivalent, but the meanings of their intercepts and coefficients related to x3 vary. There's a better approach than both. With dummy coding, you would end up with a simpler model that is easier to interpret. In the first case, you are forcing $$\beta_3 X_3$$ to be twice higher (or lower) for group coded 2 vs 1. To adapt to a specific difference between groups, the parameter, and intercept need to adapt. It's much easier to interpret the parameters using 0 and 1 codes, so $$\beta_3 \times 1$$ (vs $$0$$) serves as an additional intercept for the group coded as 1. In the second case $$X_{3a}$$ and $$X_{3b}$$ are colinear and cannot be used. What you need to do is to encode the categories as 0 and 1, so you encode the presence of one category that has an additive effect $$\beta_3 X_3 = \beta_3 1 = \beta_3$$. • forcing β3X3 To be two times higher (or lower) for group coded 2 vs 1 - Agree, but wouldn't the new slope and intercept adjust accordingly to equalize the overall effects in Y? – pqrz Jul 12, 2022 at 4:47 • What you need to do is to encode the categories as 0 and 1 - Are you also referring to one-hot encoding, which I am exactly mentioning in my Choice#2? – pqrz Jul 12, 2022 at 4:49 • @pqrz the model is linear, so$\beta_3 \times 1$will always be two times the$\beta_3 \times 2$, the parameter can't “accommodate” anyhow. As for your second comment, I refer to the dummy coding, not one-hot encoding, the difference is dropping one, redundant column. – Tim Jul 12, 2022 at 17:04 • Tim: You're right regarding choice 2, but re choice 1,multiple regression is affine invariant, meaning that if you change the coding of the$X\$ by linear transformation, you get equivalent results (adapting the intercept accordingly). Also @pqrz Jul 16, 2022 at 13:00
• @whuber fair, the wording was too strong.
– Tim
Jul 16, 2022 at 16:29

@Tim,

I think both would give same performance because:

# Read Data
data("mtcars")
data <- mtcars
attach(data)

# Model 1: My Model
am_new <- am+1
model_1_my <- lm(mpg ~ am_new)
print(summary(model_1_my))        # Output: R-squared:  0.3598

# Model 2: Tim's Model
model_2_tim <- lm(mpg ~ am)
print(summary(model_2_tim))       # Output: R-squared:  0.3598   (again)

• Dear fellows, Please let me know, what I am missing _/_
– pqrz
Jul 16, 2022 at 10:30
• I think the idea's correct, but the "best parameters" sections are puzzling, because they appear to be solving systems of linear equations rather than finding least squares solutions.
– whuber
Jul 16, 2022 at 17:09