11
$\begingroup$

What is the statistical difference between doing a linear regression in R with the formula set to y ~ x + 0 instead of y ~ x? How do I interpret those two different results?

$\endgroup$
18
$\begingroup$

Adding +0 (or -1) to a model formula (e.g., in lm()) in R suppresses the intercept. This is generally considered a bad thing to do; see:

The estimated slope is calculated differently depending on whether the intercept is estimated as well, namely:

\begin{align} \hat\beta_1 &= \frac{\sum x_iy_i - \frac{\big(\sum x_i\big)\big(\sum y_i\big)}{N}}{\sum x_i^2 - \frac{\big(\sum x_i\big)^2}{N}} \tag{with intercept} \\[15pt] \hat\beta_1 &= \frac{\sum x_iy_i}{\sum x_i^2} \tag{without intercept} \end{align}

Since the quantity to be subtracted (the "subtrahend") in both the numerator and denominator are not necessarily $0$, the estimate of the slope is biased when the intercept is suppressed.

The value for $R^2$ is also calculated differently; see:

Here are the underlying formulas:

\begin{align} R^2 &= 1 - \frac{\sum (y_i - \hat y_i)^2}{\sum (y_i - \bar y)^2} \tag{with intercept} \\[15pt] R^2 &= 1 - \frac{\sum (y_i - \hat y_i)^2}{\sum y_i^2} \tag{without intercept} \end{align}

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you, gung! If I suppress the Intercept, my multiple R-squared improves, suddenly. Can you help me out here? $\endgroup$ – JimBoy Sep 26 '15 at 20:18
  • 6
    $\begingroup$ There exists no agreed upon way to calculate r squared without an intercept. The r squared does not have its usual interpretation. Doing regression without an intercept is almost always a VERY bad idea $\endgroup$ – Repmat Sep 26 '15 at 20:26
  • $\begingroup$ @Repmat: see also stats.stackexchange.com/questions/171240/… $\endgroup$ – user83346 Sep 27 '15 at 9:22
  • 1
    $\begingroup$ @JimBoy: see stats.stackexchange.com/questions/171240/… $\endgroup$ – user83346 Sep 27 '15 at 9:23
5
$\begingroup$

It depends on context (of course), in the lm(...) command in R it will suppress the intercept. That is, you do regression though the origin.

Note that most textbook on the subject of regression, will tell you that forcing the intercept (to any value) is a bad idea.

The interpretation of x does not change, but the value (comparing with and without an intercept) will change, sometimes very significantly.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you, Repmat! I get very different estimates if I suppress the intercept compared to when I don't. In addition, all t-tests become highly significant. Do you know why this is? $\endgroup$ – JimBoy Sep 26 '15 at 20:15
  • 2
    $\begingroup$ The intercept will absorb any non 0 means variables not contained in the model. With the intercept gone, the variance has to go somewhere. This is why most book, as a general rule, states that regression without an intercept is always wrong. That is, OLS is always biased and in consistent in this case (with a few exceptions). $\endgroup$ – Repmat Sep 26 '15 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.