# Significant predictor in larger model - model significantly better?

When creating nested models, does the addition of a predictor that turns out to be significant (Pr(>|t|) < 0.05) mean that the larger model is significantly better? If not, why not?

YES, BUT...

...it depends on what exactly you mean by "significant". It is the case that the p-value from t-testing that one added feature is the same as the p-value you would get from F-testing the nested models. In that sense, you might consider the larger model to be significantly better. In terms of practical significance, though, such a p-value tells you next to nothing, as p-values can be driven small despite tiny effect sizes when the sample size is large.

Let's look at a simulation to show that the t-based and F-based p-values are the same.

set.seed(2023)
N <- 100
x1 <- runif(N)
x2 <- x1 + runif(N)
y <- x1 - 0.3*x2 + rnorm(N)
L0 <- lm(y ~ x1)
L1 <- lm(y ~ x1 + x2)
summary(L1)\$coef # t-based p-value is 0.0959419435
anova(L0, L1)    # F-based p-value is 0.09594


Aside from rounding, the p-values are equal.

• Thank you! In consequence, does it mean that the sample size calculation for a nested model comparison is the same as for the larger model alone?
– drat
Commented Jan 30, 2023 at 20:19
• The sample size is the same, yes. You just consider more features of the observations in your sample. @drat
– Dave
Commented Jan 30, 2023 at 20:24