Both models will yield exactly the same predictions -- but they might be harder to interpret than standard encodings. Let's look at them.
Please note, though, that by "binary categorical variable ... 1 or 2" we must understand that the values of X3
are treated by lm
as literally being the numbers $1$ or $2.$ If X3
is a factor variable (which is how one ordinarily would store a categorical value), then lm
will internally convert one of the values to $1$ and the other to $0$.
First model
The formula string ~ X1 + X2 + X3
implicitly includes an intercept. Mathematically -- and more explicitly -- the model is
$$E[Y] = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 X_3$$
where the $X_i$ are numerical random variables and the $\beta_i$ are the coefficients to be estimated. This gives two expressions corresponding to the two values of $X_3:$
When $X_3=1,$ the model is $E[Y] = (\beta_0 + \beta_3) + \beta_1X_1 + \beta_2 X_2.$
When $X_3=2,$ the model is $E[Y] = (\beta_0 + \beta_3) + \beta_1 X_1 + \beta_2 X_2 + \beta_3.$
I wrote these expressions in a somewhat strange way to make it clear that this model is equivalent to $$E[Y] = \alpha_0 + \beta_1 X_1 + \beta_2 X_2 + \beta_3 \mathscr{I}(X_3=2)$$ in which $\mathscr{I}(X_3=2)$ is the "one-hot encoding" for $X_3$ and, evidently, $\alpha_0 = \beta_0 + \beta_3.$
In this sense the first model is equivalent to what lm
would do with a categorical variable, but the intercept it reports (namely, $\beta_0$) will differ (in a predictable way) from the usual intercept.
Second model
I will continue to use the standard indicator notation, so that X3a
is mathematically expressed as $\mathscr{I}(X_3=1)$ and X3b
is $\mathscr{I}(X_3=2).$ This model therefore is
$$\begin{aligned}
E[Y] &= \gamma_0 + \gamma_1X_1 + \gamma_2 X_2 + \gamma_3 \mathscr{I}(X_3=1) + \gamma_4 \mathscr{I}(X_3=2)\\
&= (\gamma_0 - \delta) + \gamma_1 X_1 + \gamma_2 X_2 + (\gamma_3 + \delta)\mathscr{I}(X_3=1) + (\gamma_4 + \delta)\mathscr{I}(X_3=2).
\end{aligned}$$
The second line comes from adding and subtracting $\delta$ from the right hand side and using
$$1 = \mathscr{I}(X_3=1) + \mathscr{I}(X_3=2).$$
That is, since X3
is either 1
or 2
, the sum of the two "one-hot" codes is constantly $1.$
The point is that including all three of an intercept, X3a
, and X3b
is redundant. The statistical term for this situation is that the model is not identifiable. lm
will handle this by recognizing the redundancy by the time it processes the + X3b
term and will just drop that term, reporting its coefficient as NA
. Consequently, the lm
implementation is
$$E[Y] = \gamma_0 +\gamma_1X_1 + \gamma_2 X_2 + \gamma_3 \mathscr{I}(X_3=1).$$
As in the first model, there are two cases for the two possible values of $X_3:$
When $X_3=1,$ the model is $E[Y] = (\gamma_0 + \gamma_3) + \gamma_1X_1 + \gamma_2 X_2 + \gamma_3.$
When $X_3=2,$ the model is $ (\gamma_0 + \gamma_3) + \gamma_1X_1 + \gamma_2 X_2 - \gamma_3.$
Again I have written these to aid in the comparison with the first model. Evidently $\gamma_0 + \gamma_3 = \alpha_0$ and $\gamma_3 = -\beta_3.$
In this sense the second model is equivalent to what lm
would do with a categorical variable, but it will report a (predictably) different intercept and the coefficient of the term for $X_3$ will be negated.
Having done the analysis, let's compare with what lm
does. I created a tiny data frame (just four observations), but it's enough to yield estimates. In this model $\alpha_0 = 1,$ $\beta_1 = -2,$ $\beta_2 = -1,$ and $\beta_3 = 3.$
X <- data.frame(x1 = 1:4,
x2 = (1:4)^2,
x3 = rep(1:2, each=2))
b <- c(1,-2,-1,3)
X$y <- with(X, b[1] + b[2] * x1 + b[3] * x2 + b[4] * ifelse(x3 == 2, 1, 0))
# Default `lm` result
(lm(y ~ x1 + x2 + factor(x3), X))
# First model
(lm(y ~ x1 + x2 + x3, X))
# Second model
X$x3a <- ifelse(X$x3 == 1, 1, 0)
X$x3b <- ifelse(X$x3 == 2, 1, 0)
(lm(y ~ x1 + x2 + x3a + x3b, X))
The output for the three models lists the estimated coefficients:
lm(formula = y ~ x1 + x2 + factor(x3), data = X)
(Intercept) x1 x2 factor(x3)2
1 -2 -1 3
lm(formula = y ~ x1 + x2 + x3, data = X)
(Intercept) x1 x2 x3
-2 -2 -1 3
The estimate of $-2$ for the intercept is the difference $1-3$ (intercept minus x3
coefficient) from the first output.
lm(formula = y ~ x1 + x2 + x3a + x3b, data = X)
(Intercept) x1 x2 x3a x3b
4 -2 -1 -3 NA
The estimate of $4$ for the intercept is the sum $1+3$ (intercept plus x3
coefficient) from the first output; the estimate of $-3$ for the x3a
term is the negative of the x3
coefficient from the first output; and NA
is reported for the x3b
term.
All three models (lm
, first, and second) are equivalent, but the meanings of their intercepts and coefficients related to x3
vary.
β
's will be different - but the final loss (goodness of fit) should be same, isn't it? $\endgroup$lm
is doing in order to interpret its output. The default tests thatlm
performs will be slightly different, too, because the meanings of some of the parameters vary from one model to the next. Finally, there are far more than two ways to write an equivalent model! $\endgroup$