I'm trying to calculate partitioned sum of squares in a linear regression. In the first model, there are two predictors. In the second model, one of these predictors in removed. In the model with two predictors versus the model with one predictor, I have calculated the difference in regression sum of squares to be 2.72 - is this correct? If this is correct, why is the difference in sum of squares 2.72 (quite small) when the difference in r-squared is ~30% (quite large).

# model two predictors
mod <- lm(drat ~ hp + wt, mtcars)

# regression sum of squares two predictors
regressionSumSquares <- sum((predict(mod)-mean(mtcars$drat))*(predict(mod)-mean(mtcars$drat)))

# residuals two predictors
residualSumSquares <- sum((predict(mod)-mtcars$drat)*(predict(mod)-mtcars$drat))

# r-squared two predictors
totalSumSquares <- regressionSumSquares + residualSumSquares
rSquared <- regressionSumSquares/totalSumSquares

# model one predictor 
modJustHp <- lm(drat ~ hp, mtcars)

# regression sum of squares one predictor
regressionSumSquaresJustHp <- sum((predict(modJustHp)-mean(mtcars$drat))*(predict(modJustHp)-mean(mtcars$drat)))

# residual sum of squares one predictor
residualSumSquaresJustHp <- sum((predict(modJustHp)-mtcars$drat)*(predict(modJustHp)-mtcars$drat))

# r-squared one predictor
totalSumSquaresJustHp <- regressionSumSquaresJustHp + residualSumSquaresJustHp
rSquaredJustHp <- regressionSumSquaresJustHp/totalSumSquaresJustHp

# difference in sum of squares one vs two predictors
regressionSumSquares - regressionSumSquaresJustHp
  • $\begingroup$ I edited difference in $R^2$ to be 30%, not 30. Presume that is what you meant. $\endgroup$ – Nick Cox Jun 28 '13 at 12:23

Look at the output of this:

> var(mtcars$drat)*length(mtcars$drat)
[1] 9.148203
> sum(residuals(mod)^2)
[1] 4.35744
> sum(residuals(modJustHp)^2)
[1] 7.077585

So the null model (intercept only) has a residual sum of squares ~9. Model with 2 regressors ~4 and model with one regressor ~7. The improvement of R-squared of 0.3 from the model with one regressor to the model with 2 regressors seems quite feasible. The idea is always to put a number in a proper context.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.