Return to Answer

Let’s assume the usual nice properties we assume for linear regression parameter inference. The t-test of parameter $\beta_p$ is equivalent to F-testing the full model against the model with $\beta_p$ omitted. Let's do a simulation where that $\beta_p$ corresponds to an indicator variable of group membership: control (0) 111versus treatment (1). The full model has one continuous variable, $X_1$, and then the binary group membership variable, $X_2$.

Let’s assume the usual nice properties we assume for linear regression parameter inference. The t-test of parameter $\beta_p$ is equivalent to F-testing the full model against the model with $\beta_p$ omitted. Let's do a simulation where that $\beta_p$ corresponds to an indicator variable of group membership: control (0) versus treatment (1). The full model has one continuous variable, $X_1$, and then the binary group membership variable, $X_2$.

$$F_{p_{full}-p_{reduced},N-p_{full}}\sim F .$$

$$df_1 = p_{full}-p_{reduced}\\df_2 = N-p_{full}\\ F\sim F_{df_1,df_2} .$$

Let’s assume the usual nice properties we assume for linear regression parameter inference. The t-test of parameter $\beta_p$ is equivalent to F-testing the full model against the model with $\beta_p$ omitted. Let's do a simulation where that $\beta_p$ corresponds to an indicator variable of group membership: control (0) 111versus treatment (1). The full model has one continuous variable, $X_1$, and one binary variable, $X_2$.

set.seed(2019)
N <- 1000
beta <- c(1,-0.2,0.01)
err <- rnorm(N,0,0.1)
x1 <- rnorm(N)
x2 <- rbinom(N,1,0.5)
X <- cbind(1,x1,x2)
y <- X %*% beta + err
L_full <- lm(y~x1+x2)
#
# Now let's build a reduced model that excludes x2
#
L_reduced <- lm(y~x1)
#
# Now let's find the F-stat for the full model against the reduced model
#
sse0 <- sum(resid(L_reduced)^2)
sse1 <- sum(resid(L_full)^2)
p0 <- dim(summary(L_reduced)$coefficients)[1]
p1 <- dim(summary(L_full)$coefficients)[1]
F <- ((sse0-sse1)/(p1-p0)) / ((sse1)/(N-p1))
#
# Print the F-stat
#
print(F)
#
# Print the squared t-stat for x2 in the full model
#
print((summary(L_full)$coefficients[3,3])^2)
#
# print the p-value from the F-test
# 
print(1-pf(F,p1-p0,N-p1))
#
# print the p-value from the t-test of x2 on the full model
#
print(summary(L_full)$coefficients[3,4])
#
# F = 10.40076
# t^2 = 10.40076
# p = 0.001300581 for the F-test on 1 and 997 degrees of freedom
# p = 0.001300581 for the regular printout from R's t-test of the x2 parameter in the full model

The equation I use to calculate the F-stat comes from Agresti's Foundations of Linear and Generalized Linear Models on page 89:

$$ F_{p_{full}-p_{reduced},N-p_{full}} \sim \dfrac{(SSE_{reduced}-SSE_{full})/(p_{full}-p_{reduced})}{SSE_{full}/(N-p_{full})} .$$

Let’s assume the usual nice properties we assume for linear regression parameter inference. The t-test of parameter $\beta_p$ is equivalent to F-testing the full model against the model with $\beta_p$ omitted. Let's do a simulation where that $\beta_p$ corresponds to an indicator variable of group membership: control (0) 111versus treatment (1). The full model has one continuous variable, $X_1$, and then the binary group membership variable, $X_2$.

set.seed(2019)
N <- 1000
beta <- c(1,-0.2,0.01)
err <- rnorm(N,0,0.1)
x1 <- rnorm(N)
x2 <- rbinom(N,1,0.5)
X <- cbind(1,x1,x2)
y <- X %*% beta + err
L_full <- lm(y~x1+x2)
#
# Now let's build a reduced model that excludes x2
#
L_reduced <- lm(y~x1)
#
# Now let's find the F-stat for the full model against the reduced model
#
sse0 <- sum(resid(L_reduced)^2)
sse1 <- sum(resid(L_full)^2)
p0 <- dim(summary(L_reduced)$coefficients)[1]
p1 <- dim(summary(L_full)$coefficients)[1]
F <- ((sse0-sse1)/(p1-p0)) / ((sse1)/(N-p1))
#
# Print the F-stat
#
print(F)
#
# Print the squared t-stat for x2 in the full model
#
print((summary(L_full)$coefficients[3,3])^2)
#
# print the p-value from the F-test
# 
print(1-pf(F,p1-p0,N-p1))
#
# print the p-value from the t-test of x2 on the full model
#
print(summary(L_full)$coefficients[3,4])
#
# F = 10.40076
# t^2 = 10.40076
# p = 0.001300581 for the F-test on 1 and 997 degrees of freedom
# p = 0.001300581 for the regular printout from R's t-test of beta2 in the full model

As the last four lines show, the F-stat is equal to the squared t-stat, and, when the F-test has the appropriate degrees of freedom, both the F-test of full versus reduced and the t-test of $\beta_2$ give identical p-values.

The equation I use to calculate the F-stat comes from Agresti's Foundations of Linear and Generalized Linear Models on page 89:

$$ F= \dfrac{(SSE_{reduced}-SSE_{full})/(p_{full}-p_{reduced})}{SSE_{full}/(N-p_{full})} .$$

Agresti also gives the distribution under the null hypothesis that the reduced model holds (that is, the treatment is zero).

$$F_{p_{full}-p_{reduced},N-p_{full}}\sim F .$$