12
$\begingroup$

In my econometric textbook(Introductory Econometrics) covering OLS, the author write, "SSR must fall when another explanatory variable is added." Why is it?

$\endgroup$
3
  • 1
    $\begingroup$ In essence because if there's no linear relationship with the next variable whatever (0 sample partial correlation), the SSR will stay the same. If there's any relationship at all, the next variable can be used to reduce SSR. $\endgroup$
    – Glen_b
    Commented Aug 19, 2015 at 9:29
  • 3
    $\begingroup$ The statement is correct in spirit but not quite true: SSR will stay the same (and not fall) upon adding any variable that is a linear combination of the existing variables. After all, by ignoring the new variable you can achieve the same minimum value of SSR you accomplished with the old variable, so adding a new variable can never make things worse. $\endgroup$
    – whuber
    Commented Aug 19, 2015 at 12:43
  • $\begingroup$ I answered a similar question here: stats.stackexchange.com/questions/306267/…. You may find it useful. $\endgroup$
    – Josh
    Commented Dec 6, 2017 at 16:54

2 Answers 2

25
$\begingroup$

Assuming you have a linear regression model, for easy notation consider first one then two covariables. This generalizes to two sets of covariables. The first model is $$ I \colon y_i=\beta_0 + \beta_1 x_{1i}+\epsilon_{i} $$ the second model is $$ II \colon y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + \epsilon_i $$ This is solved by minimizing sum of squared residuals, for model one we want to minimize $\text{SSR}_1 = \sum_i (y_i-\beta_0-\beta_1 x_{1i})^2$ and for model two you want to minimize $\text{SSR}_2 = \sum_i (y_i-\beta_0-\beta_1 x_{1i}-\beta_2 x_{2i})^2$. Lets say you have found the correct estimators for model 1, then you can obtain that exact same residual sum squares in model two by choosing the same values for $\beta_0, \beta_1$ and letting $\beta_2=0$. Now you can find, possibly, a lower sum squares residual by searching for a better value for $\beta_2$.

To summarize, the models are nested, in the sense that everything we can model with model 1 can be matched by model two, model two is more general than model 1. So, in the optimization, we have larger freedom with model two so can always find a better solution.

This has really nothing to do with statistics but is a general fact about optimization.

$\endgroup$
1
  • 1
    $\begingroup$ Haven't thought in this way, really helpful! $\endgroup$
    – Eric Xu
    Commented Aug 19, 2015 at 8:30
2
$\begingroup$

SSR is a measure of the discrepancy between the data and an estimation model.

If you have the option to take into account another variable , then if this variable contains more information, the fit would naturally be tighter, which means a lower SSR.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.