What is the statistical difference between doing a linear regression in R with the formula set to y ~ x + 0 instead of y ~ x? How do I interpret those two different results?
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
Adding +0 (or -1) to a model formula (e.g., in lm()) in R suppresses the intercept. This is generally considered a bad thing to do; see:
- When is it OK to remove the intercept in lm()?
- When forcing intercept of 0 in linear regression is acceptable/advisable
The estimated slope is calculated differently depending on whether the intercept is estimated as well, namely:
\begin{align} \hat\beta_1 &= \frac{\sum x_iy_i - \frac{\big(\sum x_i\big)\big(\sum y_i\big)}{N}}{\sum x_i^2 - \frac{\big(\sum x_i\big)^2}{N}} \tag{with intercept} \\[15pt] \hat\beta_1 &= \frac{\sum x_iy_i}{\sum x_i^2} \tag{without intercept} \end{align}
Since the quantity to be subtracted (the "subtrahend") in both the numerator and denominator are not necessarily $0$, the estimate of the slope is biased when the intercept is suppressed.
The value for $R^2$ is also calculated differently; see:
- Removal of statistically significant intercept term boosts $R^2$ in linear model
- How can R2 have two different values for the same regression (without an intercept)
Here are the underlying formulas:
\begin{align} R^2 &= 1 - \frac{\sum (y_i - \hat y_i)^2}{\sum (y_i - \bar y)^2} \tag{with intercept} \\[15pt] R^2 &= 1 - \frac{\sum (y_i - \hat y_i)^2}{\sum y_i^2} \tag{without intercept} \end{align}
answered Sep 26 '15 at 20:01
gung - Reinstate Monica♦gung - Reinstate Monica
121k40 gold badges314 silver badges603 bronze badges
-
$\begingroup$ Thank you, gung! If I suppress the Intercept, my multiple R-squared improves, suddenly. Can you help me out here? $\endgroup$ – JimBoy Sep 26 '15 at 20:18
-
6$\begingroup$ There exists no agreed upon way to calculate r squared without an intercept. The r squared does not have its usual interpretation. Doing regression without an intercept is almost always a VERY bad idea $\endgroup$ – Repmat Sep 26 '15 at 20:26
-
$\begingroup$ @Repmat: see also stats.stackexchange.com/questions/171240/… $\endgroup$ – user83346 Sep 27 '15 at 9:22
-
1$\begingroup$ @JimBoy: see stats.stackexchange.com/questions/171240/… $\endgroup$ – user83346 Sep 27 '15 at 9:23
$\begingroup$
$\endgroup$
It depends on context (of course), in the lm(...) command in R it will suppress the intercept. That is, you do regression though the origin.
Note that most textbook on the subject of regression, will tell you that forcing the intercept (to any value) is a bad idea.
The interpretation of x does not change, but the value (comparing with and without an intercept) will change, sometimes very significantly.
-
$\begingroup$ Thank you, Repmat! I get very different estimates if I suppress the intercept compared to when I don't. In addition, all t-tests become highly significant. Do you know why this is? $\endgroup$ – JimBoy Sep 26 '15 at 20:15
-
2$\begingroup$ The intercept will absorb any non 0 means variables not contained in the model. With the intercept gone, the variance has to go somewhere. This is why most book, as a general rule, states that regression without an intercept is always wrong. That is, OLS is always biased and in consistent in this case (with a few exceptions). $\endgroup$ – Repmat Sep 26 '15 at 20:20
