regression coefficient on sum of regressors

Say I have the true linear model with normal errors:

$y = \beta_1 X_1 + \beta_2 X_2 + \epsilon$

However, I only observe $Z = X_1 + X_2$, so I estimate instead:

$y = \delta (X_1 + X_2) + e$,

can I express $\delta$ in terms of $\beta_1$ and $\beta_2$ (and perhaps $X_1$ and $X_2$)?

Observably when I generate data and estimate the results $\delta$ seems to be the average of the coefficients weighted by the variance of the regressors, but I can't seem to derive a solution.

Incorporating hints:

Thanks for the hints, which are helpful. Here is an attempt -- although I feel I haven't gotten to the bottom of it yet:

$y = a(X_1 + X_2) + b(X_1 - X_2) + \epsilon$ (thanks @whuber!)

$y = aX_1 + aX_2 + bX_1 - bX_2 + \epsilon$,

$y = (a + b)X_1 + (a - b)X_2 + \epsilon$, therefore

$\beta_1 = (a+b)$ and $\beta_2 = (a - b)$.

Solving for $a$ and $b$ we get:

$a = (\beta_1 + \beta_2)/2$ and $b = (\beta_1 - \beta_2)/2$

Effectively I am estimating $a$ while omitting $b$, so

$y = a(X_1 + X_2) + u,$

$u = b(X_1 - X_2) + \epsilon$, therefore my estimate of $\delta$ above will be the estimate of $a$ with omitted variable bias. Calling $X_1 + X_2 = Z$, my estimate of $\hat{\delta}$ is therefore:

$\hat{\delta} = (Z'Z)^{-1}(Z'[Za + (X_1 - X_2)b + \epsilon])$

Ignoring $\epsilon$ as it disappears in expectation, I get

$\hat{\delta} = a + (Z'Z)^{-1}(Z'(X_1 - X_2))b$

$\hat{\delta} = \frac{\beta_1 + \beta_2}{2} + (Z'Z)^{-1}Z'[X_1 - X_2] \frac{\beta_1 - \beta_2}{2}$

Does this look on the right track? Can we reduce the trailing second expression?

I was thinking that since the expression $(Z'Z)^{-1}Z'[X_1 - X_2]$ looks like a regression of $X_1 - X_2$ on $Z$, I could perhaps re-write it as $Var(X_1 + X_2)^{-1}Cov(X_1 + X_2, X_1 - X_2) = Var(X_1+X_2)^{-1}(Var(X_1) - Var(X_2))$?

• Hint: Comparing your model to the original one as written in the form $$y = \alpha(X_1+X_2) + \beta(X_1-X_2)+\epsilon$$ is routine and your model is the case $\beta=0.$ – whuber Apr 19 '18 at 13:24
• Another hint: you can't do it only with $\beta_1$ and $\beta_2$, you need information about the variance of these estimates – Richard Border Apr 19 '18 at 14:26
• Thanks for the hints @whuber and Richard Border, I made some edits to reflect progress. The expression I derived still looks a little complicated -- is there a simpler way of thinking about it? – gfgm Apr 19 '18 at 15:35
• Yes, there is a simpler way: this multiple regression can be accomplished in two stages of simple regression, as described at stats.stackexchange.com/questions/46185/…. – whuber Apr 19 '18 at 15:44
• Thanks @whuber the link was helpful, I think I found the easier way your were indicating. Is the citation you are referencing in the linked post the book Data Analysis and Regression? – gfgm Apr 20 '18 at 8:58

With a great deal of prodding (full credit to @whuber) I seem to have solved it:

We re-write $y = \beta_1X_2 + \beta_2X_2 + \epsilon$ as

$y = a(X_1 + X_2) + b(X_1 - X_2) + \epsilon$

$y = aX_1 + aX_2 + bX_1 - bX_2 + \epsilon$,

$y = (a + b)X_1 + (a - b)X_2 + \epsilon$, therefore

$\beta_1 = (a+b)$ and $\beta_2 = (a - b)$.

Solving for $a$ and $b$ we get:

$a = (\beta_1 + \beta_2)/2$ and $b = (\beta_1 - \beta_2)/2$

What I estimate is equivalent to

$y = \delta(X_1 + X_2) + u,$

$u = b(X_1 - X_2) + \epsilon$. We know that the $k$th coefficient of a multiple regression can be recovered by first partialling out the regressors other than $k$. Thus, if I let $z = X_1 + X_2$ and $q = X_1 - X_2$, the regression coefficient $b$ in:

$y = az + bq + \epsilon$ can be obtained with the steps

(1) $y = \delta z + u$,

(2) $q = \lambda z + e$

(3) $u = be + \epsilon$. Substituting expressions back in,

$y = \delta z + be + \epsilon$ from (1).

$y = \delta z + b(q - \lambda z) + \epsilon$ from (2). Therefore:

$y = (\delta - b \lambda)z + bq + \epsilon$. Now we know that $(\delta - b\lambda ) = a = (\beta_1 + \beta_2)/2$, so we solve for $\delta$ which yields:

$\delta = (\beta_1 + \beta_2)/2 + \lambda (\beta_1 - \beta_2)/2$

Here is some R code to check the solution, or to play around with:

rep.func <- function(N  = 100, b1 = 2, b2 = 10, s1 = 1, s2 = 5) {
x1 <- rnorm(N, 0, s1)
x2 <- rnorm(N, 0, s2)
eps <- rnorm(N)
y <- x1*b1 + x2*b2 + eps
z <- x1 + x2
q <- x1 - x2
lambda <- lm(q ~ z - 1)$coefficients est.delta <- lm(y ~ z - 1)$coefficients
est.betas <- lm(y ~ x1 + x2 - 1)\$coefficients
derived.delta <- sum(est.betas)/2 + lambda * (est.betas - est.betas)/2
c("est.delta" = est.delta, "derived.delta" = derived.delta)
}

rep.func()
#>     est.delta.z derived.delta.z
#>         9.77236         9.77236

library(ggplot2)
res <- t(replicate(1000, rep.func()))

all.equal(res[,1], res[,2])
#>  TRUE

ggplot(as.data.frame(res), aes(est.delta.z, derived.delta.z)) +
geom_point() + geom_smooth(method='lm') + theme_minimal() 