# Linear Regression: P-values of t-test of significance of regression coeficients are same as p-values of F-test about submodel

Suppose we have regression problem:

$$Y = \beta_{0}+\beta_{1}X_{1} + \beta_{2}X_{2} + \epsilon \text{, } \epsilon \sim \mathcal{N}(0,1),$$

where $$X_1 \sim U(0,1)$$,$$X_{2} \sim U(1,2)$$, and we suppose our model with regression function

$$m:=E[Y|X_1,X_2] = \beta_{0}+\beta_{1}X_{1} + \beta_{2}X_{2}.$$

We apply OLS estimation of coefficient and receive estimate of regression coeficeint. Let's do this in $$\verb|R|$$:

X1 <- runif(1000,0,1)
X2 <- runif(1000,1,2)
Y <- X1 + X2 + rnorm(1000,0,1)
model <- lm(Y ~ X1 + X2)
summary(model)

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)     0.1401     0.1828   0.766    0.444
X1              1.0250     0.1150   8.909  < 2e-16 ***
X2              0.9247     0.1139   8.119 1.37e-15 ***
---
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.026 on 997 degrees of freedom
Multiple R-squared:  0.1276,    Adjusted R-squared:  0.1259
F-statistic: 72.94 on 2 and 997 DF,  p-value: < 2.2e-16


Next, we want to test, if the regression coeficient $$\beta_2$$ is significant or not, so here is T-test for this:

$$H_{0}: \beta_{2} = 0$$ $$H_{1}: \beta_{2} \neq 0$$

Aproppiate test statistic is:

$$T_n = \frac{\hat{\beta_{2}}}{S.E(\hat{\beta_{2}})} \sim t_{n-r},$$

and p-value:$$2min\{CDF_{t}(t_{0}), 1 - CDF_{t}(t_0)\}$$, where $$CDF_t$$ is cummulative distribution function o $$t$$-distribution with $$n-r$$, in ou case $$1000-3$$ degrees of freedom. Let's suppose, that we also made test about submodel:

$$H_{0} Y \sim X_{1} \text{ holds} (M^{0})$$ $$H_{1} Y \sim X_{1} + X_{2} \text{ holds} (M).$$

Test statistics is $$F = \frac{\frac{SSe^0 - SSe}{r - r_{0}}}{MSe} \sim F_{r-r_0,n-r}$$ and p-value:$$1-CDF_f(f_0)$$, where $$CDF_f$$ is cummulative distribution function of F-distribution with $$r-r_0$$ and $$n-r$$ degrees of freedom. In $$\verb|R|$$ let's make an test about submodel:

m0 <- lm(Y ~ X1)
anova(m0,model)

Analysis of Variance Table

Model 1: Y ~ X1
Model 2: Y ~ X1 + X2
Res.Df    RSS Df Sum of Sq      F    Pr(>F)
1    998 1118.2
2    997 1048.8  1    69.352 65.926 1.373e-15 ***
---
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1


Now there is a question, i know that if $$X \sim t_{n-r}$$, then $$X^2 \sim F_{1,n-r}$$. So if we squared t-value in our summary table, in line where regression coeficient $$\hat{\beta}_2$$ is, we geta f-value in output in test of submodel, so that's okay, but we have also same p-values, but p-value in summary table is computed from t-distribution, not squared distribution, and this numbers implies that $$2min\{CDF_{t}(t_{0}), 1 - CDF_{t}(t_0)\} = 1-CDF_f(f_0)$$, but i think this is not really true. What am i missing, how it is possible, that p-value which are computed from two different distributions are same ? (and this is not only for this case, but for every liner regression problems). Why p-values are same ? Please help.

In all cases i asuume normal linear model, that's why i assume those distribution of test statistics under $$H_0$$ hypothesis.

You should expect that the p-values are the same. If you reject the null hypothesis that $$\beta_2$$ = 0 (using the t-test), you should simultaneously reject the null hypothesis that the model without $$\beta_2$$ is adequate (using the F-test).

The p-value for the t-test is the probability that $$|t_0| > t^{\alpha/2}_{n-r}$$, and the p-value for the F-test is the probability that $$F_0 > F^{\alpha/2}_{1, n-r}$$. But note that these are mathematically identical statements, since squaring both sides of $$|t_0| > t^{\alpha/2}_{n-r}$$, gives you $$F_0 > F^{\alpha/2}_{1, n-r}$$, as you stated. Therefore, the p-values will always be numerically identical in this situation.

It's true that the p-value is computed from two different distributions, but they're found using different test statistics which are related to one another in a way that the p-value will always be identical.

• this would be a good answer if you include how are the two test statistics related to each other and why will they always (or almost always?) produce the same p-value Jan 8 '20 at 17:13
• Thanks for your feedback @rep_ho. I updated my answer with some more details on how the statistics are related to one another. Jan 8 '20 at 18:14
• Yes, this is right, thank you for your help. :) Jan 9 '20 at 13:04