I have a question about testing the equivalence of two regression parameters. I know there are a lot of resources about how to test a linear hypothesis but I am confused about my results (and its been a long time since I’ve did my university stat subjects).

I have a simple linear model of annual earnings (the dependent variable) regressed on treatment variables (participants received either treatment 1 or treatment 2) plus some control variables. $$ earnings = \beta_1 *treat1 + \beta_2 *treat2 + \beta_{[3-6]} *controls $$ There are 30,451 observations in the data. The OLS model output is shown below.

Output of unrestricted model

I would like to test if the treatment effect of the two treatments are equivalent. That is: $$ H_0: \beta_1 = \beta_2 \\ H_1: \beta_1 \neq \beta_2 $$ I know I can do this through a Wald test. Specifically I can compare the fit of the restricted and unrestricted models. When I impose the restriction and re-estimate the model I get the following.

Output of restricted model

I then calculate the F-statistic using this output: $$ F = \frac{\frac{R^2_{u} – R^2_{r}}{J}}{\frac{1-R^2_u}{n-k}} = \frac{0.7280015 - 0.727763}{\frac{- 0.727763}{30445}} = 26.68 $$ Looking at the F dist with F(1, 30445) I get this being basically zero. I reject the hypothesis. (I get the same results when using linearHypothesis() from the car package in R)

However, rejecting the hypothesis is incredibly surprising. The difference between the two estimates is only about 3000, and the standard errors on both these estimates is sizeable in comparison (about 2200). Intuition says they are not statistically difference. Indeed, if I was to simply test if treat1 = 53473.41 (the value of treat2) I get $ t = -1.43 $ and fail to reject.

I’m clearly misunderstanding something here. Can anyone tell me where I am going wrong? And more to the point, what procedure should I use to test the hypothesis that my two treatment coefficients are equal?

  • $\begingroup$ Wouldn't the usual factor coding for treatment provide exactly the desired test result? I.e. using formula y ~ treat + control1 + control2 ... $\endgroup$ – Michael M Jul 24 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.