Skip to main content
2 of 6
added RSS
Ernest A
  • 2.4k
  • 3
  • 19
  • 18

Removal of statistically significant intercept term boosts R^2 in linear model

In a simple linear model with a single explanatory variable,

$\alpha_i = \beta_0 + \beta_1 \delta_i + \epsilon_i$

I find that removing the intercept term improves the fit greatly (value of $R^2$ goes from 0.3 to 0.9). However, the intercept term appears to be statistically significant.

With intercept:

Call:
lm(formula = alpha ~ delta, data = cf)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.72138 -0.15619 -0.03744  0.14189  0.70305 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.48408    0.05397    8.97   <2e-16 ***
delta        0.46112    0.04595   10.04   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.2435 on 218 degrees of freedom
Multiple R-squared: 0.316,    Adjusted R-squared: 0.3129 
F-statistic: 100.7 on 1 and 218 DF,  p-value: < 2.2e-16

Without intercept:

Call:
lm(formula = alpha ~ 0 + delta, data = cf)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.92474 -0.15021  0.05114  0.21078  0.85480 

Coefficients:
      Estimate Std. Error t value Pr(>|t|)    
delta  0.85374    0.01632   52.33   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.2842 on 219 degrees of freedom
Multiple R-squared: 0.9259,   Adjusted R-squared: 0.9256 
F-statistic:  2738 on 1 and 219 DF,  p-value: < 2.2e-16

How would you interpret these results? Should an intercept term be included in the model or not?

Edit

Here's the residual sums of squares:

RSS(with intercept) = 12.92305
RSS(without intercept) = 17.69277
Ernest A
  • 2.4k
  • 3
  • 19
  • 18