I have a multiple linear regression with a couple of independent variables in it. Most of them are significant at p<0.001. The model has an R² of 0.83. When I add more variables, the old and the new variables are all highly significant, but R² does not improve at all.
What does that tell me?
No, I don't think you should be concerned about the R-squared directly. Here's an example.
R squared must be increasing, but because of precision, you might not be seeing it.
First generate some data:
library(MASS)
sigma <- matrix(c(1.0, 0.8, 0.8, 0.4,
0.8, 1.0, 0.7, 0.4,
0.8, 0.7, 1.0, 0.4,
0.4, 0.4, 0.4, 1.0),nrow=4)
d <- as.data.frame(mvrnorm(Sigma=sigma, n=2000, mu=rep(0, 4)))
names(d) <- c("y", "x1", "x2", "x3")
Run two models, one with one additional predictor.
> model1 <- lm(y ~ x1 + x2, data=d)
> model2 <- lm(y ~ x1 + x2 + x3, data=d)
> summary(model1)
Call:
lm(formula = y ~ x1 + x2, data = d)
Residuals:
Min 1Q Median 3Q Max
-1.76599 -0.32031 -0.00252 0.31977 1.58157
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.008183 0.010902 0.751 0.453
x1 0.475810 0.015359 30.980 <2e-16 ***
x2 0.470222 0.015263 30.808 <2e-16 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.4873 on 1997 degrees of freedom
Multiple R-squared: 0.7615, Adjusted R-squared: 0.7613
F-statistic: 3188 on 2 and 1997 DF, p-value: < 2.2e-16
> summary(model2)
Call:
lm(formula = y ~ x1 + x2 + x3, data = d)
Residuals:
Min 1Q Median 3Q Max
-1.6898 -0.3148 0.0086 0.3269 1.5480
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.007822 0.010861 0.720 0.471
x1 0.464192 0.015573 29.808 < 2e-16 ***
x2 0.460004 0.015417 29.837 < 2e-16 ***
x3 0.048184 0.012008 4.013 6.22e-05 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.4855 on 1996 degrees of freedom
Multiple R-squared: 0.7634, Adjusted R-squared: 0.7631
F-statistic: 2147 on 3 and 1996 DF, p-value: < 2.2e-16
In the first model, R-squared is 0.76, in the second model, R-squared is 0.76, but the p-value on x3, which was added in the second model is highly significant.
You can test the change in R-squared with the ANOVA command:
> anova(model1, model2)
Analysis of Variance Table
Model 1: y ~ x1 + x2
Model 2: y ~ x1 + x2 + x3
Res.Df RSS Df Sum of Sq F Pr(>F)
1 1997 474.26
2 1996 470.46 1 3.7953 16.102 6.223e-05 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
The p-value is the same as the p-value for x3 in the second model. The change in R-squared was small, but it was significant. That can happen, it's not necessarily a problem.
This is because at a certain point you are adding independent variables which explain similarly your response variable. In this situation, these variables are possibly multi collinear.
Example:
A model that has a certain circumference area as a function of independent variables like circumference's diameter and circumference's perimeter would probably perform the same way as if the regression model was dependent on just one of these independent variables.
