I am carrying a linear regression on some data. One of my variables is a factor (categorical). Using regression with an intercept leads to difficult interpretation, since one of the factor levels is taken as the intercept, and the remaining levels are given relative to that. Removing the intercept give me an effect of each level of the factor, which I what I want.

As far as I know, both models are precisely equivalent. They produce identical predictions (on the training set) - to within ~3e-15. However, their R² scores vary wildly.

int <- lm(fscore ~ 1 + partner.status + conformity + fcategory,
          data = Moore)  #with intercept
nint <- lm(fscore ~ 0 + partner.status + conformity + fcategory,
           data = Moore) #w/o intercept
summary(nint)$r.squared  #R² values are not remotely the same
max(predict(int)-predict(nint)) #Predictions are essentially identical

Why are the models not identical? Is it because R² is a comparision of the model to "no model", and that "no model" corresponds to "y=0" and "y=mean(fscore)", for nint and int, respectively?

  • $\begingroup$ You answered your own question in a rather imprecise manner. R^2 is not about "power" which as a precise meaning in statistics. It is about proportion of the variance explained. Incidentally, that require(car) call is superfluous. You used no 'car' functions. $\endgroup$ – DWin Jan 14 '13 at 10:02
  • $\begingroup$ The dataset "Moore" is from "car", hence the requirement. That was the most convenient dataset I could think of. Word "power" removed. $\endgroup$ – dynamo Jan 14 '13 at 10:11
  • $\begingroup$ Sorry. Did search, but presumerably I searched using the wrong words. R and statisticas question are, for whatever reason, far from uncommon on stackoverflow. Feel free to migrate me. I agree that this makes more sense on CVal. And yes - it turns out that this was not a coding question. But I did not know that; the solution could well have been to know about some "perk" in R. $\endgroup$ – dynamo Jan 14 '13 at 10:42

Seems like "thinking aloud" on Stackoverflow brings clarity to the mind! This site has a very clear demonstration of the effect: http://www.ats.ucla.edu/stat/mult_pkg/faq/general/noconstant.htm

Also, investigating the description for "r.squared" in the function documentation explains what it does.

?summary.lm #Value::r.squared
#r.squared: R^2, the 'fraction of variance explained by the model',
#              R^2 = 1 - Sum(R[i]^2) / Sum((y[i]- y*)^2),          
#           where y* is the mean of y[i] if there is an intercept and
#           zero otherwise.
| cite | improve this answer | |
  • $\begingroup$ Precicely. summary.lm gives the uncentered R-squared for models without intercept and the centered R-squared otherwise, as is appropriate. $\endgroup$ – Roland Jan 14 '13 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.