Consider the unrestricted multiple linear regression model:

$$ Y_i= \beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_3 +... + \beta_kX_k + \epsilon_i (1) $$ If we test the hypothesis $$H_0: \beta_1 + \beta_2 = 0$$ Then the restricted model when $H_0$ is true is: $$ Y_i= \beta_0 + \beta_1X_1 - \beta_1X_2 + \beta_3X_3 +... + \beta_kX_k + \epsilon_i (2) $$ So what is the restricted model when we test $$H'_0: \beta_1 + \beta_2 = c, c\neq0$$ Is it still $(1)$ but now that the restriction is the sum of two coefficients has to be $c$?

EDIT: The restricted model $(2)$ is corrected to take into account whuber's comment. Following that, the restricted model under $H'_0$ is

$$ Y_i= \beta_0 + \beta_1X_1 + (c - \beta_1)X_2 + \beta_3X_3 +... + \beta_kX_k + \epsilon_i (3) $$

I also want to ask what we can say about residual sum of squares between the restricted vs. unrestricted models in this situation? It's said that $RSS_{restricted} \ge RSS_{unrestricted}$. But in this case I'm not sure if that relationship still holds.

  • 1
    $\begingroup$ Your initial conclusion is incorrect. The restricted model under $H_0$ is $$Y_i=\beta_0 + \beta_1 X_1 - \beta_1 X_2 + \beta_3 X_3 + \cdots + \beta_k X_k + \epsilon_i.$$ This is obtained algebraically by substituting $\beta_2 = 0-\beta_1.$ To answer your question, then, consider what happens when you replace "$0$" by "$c$." $\endgroup$ – whuber Mar 15 '18 at 14:18
  • $\begingroup$ Thanks @whuber, I amended it. I guess I can see the construction of the restricted model. Would you mind addressing my question about RSS? $\endgroup$ – NonSleeper Mar 16 '18 at 1:15
  • $\begingroup$ That's trivial when you look at it the right way: what happens to the optimal value of the objective function (sum of squares of residuals) when you impose constraints on the possible solutions? $\endgroup$ – whuber Mar 16 '18 at 14:47
  • $\begingroup$ @whuber I think I didn't get it. I think what we want to look at is SSR of two models in general rather than just their optimal values, since we're concerned with comparison of these two functions, instead of the process of parameter estimation. I've played with formulas a bit but seemed to run in a loop. For example, $RSS_R \ge RSS_U$ implies $ESS_R \le ESS_U$ (ESS=explained sum of squares), or ${(Y_{iR}-\hat{y}_{iR}})^2 \le {(Y_{iU}-\hat{y}_{iU})}^2$. Then again, I find it easier to make sense of the math when null hypothesis imposes coefficient(s) to be 0. $\endgroup$ – NonSleeper Mar 17 '18 at 19:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.