Additive property of the regression coefficients (slopes)

Question

If $y_1$, $y_2$, and $y_3$ are time series such that:

$$y_1=y_2+y_3$$

Suppose all those variables were regressed against index $x$, so we get coefficients $r_{y_1}$, $r_{y_2}$, and $r_{y_3}$. In general, it should then hold that:

$$r_{y_1}=r_{y_2}+r_{y_3}$$

I expect they should not equate since variance itself is not additive, as discussed in this post here. If so, does that mean that regression is not a linear operator?

Answer 1

Ordinary linear regression

Regression coefficients are computed using a linear sum of the observations, you can write it as a matrix multiplication.

$$\hat{\beta} = M \cdot y$$

where $M = (X^TX)^{-1}X^T$. Therefore you will get

$$\hat{\beta}_{y_1} = M \cdot (y_2+y_3) = M \cdot y_2 + M \cdot y_3 = \hat{\beta}_{y_2} + \hat{\beta}_{y_3}$$

Other types of regression

For other types of regression this distributive property does not need to hold.

---

Code demonstration

set.seed(1)

x = 1:10
y2 = rexp(10,1/x)
y3 = rexp(10,1/x)
y1 = y2 + y3

glm(y1 ~ x, family = Gamma)$coefficients
glm(y2 ~ x, family = Gamma)$coefficients +
glm(y3 ~ x, family = Gamma)$coefficients
#(Intercept)           x 
# 0.140085438 -0.007893727 
#(Intercept)           x 
# 0.62397301 -0.03852544 

lm(y1 ~ x)$coefficients
lm(y2 ~ x)$coefficients +
lm(y3 ~ x)$coefficients
#(Intercept)           x 
#  5.8173249   0.9380105 
#(Intercept)           x 
#  5.8173249   0.9380105 

Answer 2

Say you have $n$ observations. Regressing $y$ on $x$ is equivalent to finding the orthogonal projection of the $n$ dimensional vector of observed values $\vec{y}$ onto the subspace of $\mathbb{R}^n$ spanned by $\vec{1}$ and the vector of observed values $\vec{x}$. The coefficient of $\vec{1}$ is the constant term and the coefficient of $\vec{x}$ is the slope.

Since orthogonal projection is a linear operator, it is true that $\textrm{Proj}_X(\vec{y_2} + \vec{y_3}) = \textrm{Proj}_X(\vec{y_2}) + \textrm{Proj}_X(\vec{y_3})$. This implies that the coefficient of $x$ in the regression of $y_1$ on $x$ will be the sum of the coefficients of $y_2$ and $y_3$ when regressed on $x$.