From my Stata experiment, the following two approaches give the same coefficient and standard errors for X.

(1) regressing Y1-Y2 on X and Y2.

(2) regressing Y1 on X and Y2.

That is,

(1) gen Ydiff = Y1-Y2 
    reg Ydiff X Y2

(2) reg Y1    X Y2

Is this generally true? Why would that be the case?

  • $\begingroup$ Hint: there's also a simple obvious relationship between the coefficient estimates for $Y_2$. $\endgroup$ – whuber Nov 7 '20 at 20:31
  • $\begingroup$ @whuber Can I safely generalize that #1 and #2 produce the same coefficient and standard errors for X? $\endgroup$ – user1849133 Nov 9 '20 at 1:09
  • $\begingroup$ @whuber Hi, I would like to revisit the question. Could you explain more? $\endgroup$ – user1849133 Jan 15 at 20:42

Yes, this is generally true.

Given data $(x_i, y_{1i}, y_{2i}),$ the second model supposes

$$y_{1i} = \beta_0 + \beta_1 x_i + \beta_2 y_{2i} + \varepsilon_i$$

for fixed unknown $\beta_j$ (which are estimated by reg). The first model supposes

$$y_{1i} - y_{2i} = \alpha_0 + \alpha_1 x_i + \alpha_2 y_{2i} + \delta_i.\tag{**}$$

By subtracting $y_{2i}$ from both sides of the first model you obtain the equivalent model

$$y_{1i}-y_{2i} = \beta_0 + \beta_1 x_i + (\beta_2-1) y_{2i} + \varepsilon_i.\tag{*}$$

Comparing this term-by-term to the first model shows $(*)$ and $(**)$ are the same but merely use different notation with

$$\alpha_0 = \beta_0,\ \alpha_1=\beta_1,\ \alpha_2 = \beta_2 - 1,\ \text{ and }\delta_i=\varepsilon_i.$$

Consequently the model estimates will have the same relationships; in particular, $\hat\alpha_0 = \hat\beta_0$ (the estimated intercept) and $\hat\alpha_1=\hat\beta_1$ (the estimated slopes) must be equal whenever the estimates are unique (which is the case when the $x_i$ are not constant). (You will also notice that $\hat\alpha_2 = \hat\beta_2 - 1.$)

Furthermore, the residuals will be the same and will have the same meanings, so the software will report that the root mean squared errors are the same. That causes all the standard errors of the parameter estimates to be the same in the two models.

However, the software will likely report a different $R^2,$ because that depends also on the variance of the responses and there's no assurance the variance of the $y_{1i}$ (in the second model) will equal the variance of the $y_{1i}-y_{2i}$ (in the second model).

  • $\begingroup$ Thank you so much! $\endgroup$ – user1849133 Jan 16 at 7:35

This is definitely not true in general.

The obvious case where this is true is when the Y2 is all zeros, then your equations are equal to each other.

x = [1, 2, 3, 4, 5, 6, 7, 8, 9]
y1 = x
y2 = x - x

# coef for x is 1 in both cases

And the obvious case where this is not true, set Y1 == Y2 (both non-zero) then in the first case, your dependent variable is all 0s, therefore the coefficient for X has to be 0, but in the second case, it won't be all 0s.

x = [1, 2, 3, 4, 5, 6, 7, 8, 9]
y1 = x
y2 = x

# coef for x is 
#   eq(1): 0
#   eq(2): 0.5
  • $\begingroup$ I am sorry I missed a critical thing in my question. (1) is regressing Y1-Y2 on X and Y2. Not just on X. Then I suppose (1) and (2) give the same coef and s.e. for X? $\endgroup$ – user1849133 Nov 7 '20 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.