1
$\begingroup$

Suppose we have regression problem:

$$Y = \beta_{0}+\beta_{1}X_{1} + \beta_{2}X_{2} + \epsilon \text{, } \epsilon \sim \mathcal{N}(0,1),$$

where $X_1 \sim U(0,1)$,$X_{2} \sim U(1,2)$, and we suppose our model with regression function

$$m:=E[Y|X_1,X_2] = \beta_{0}+\beta_{1}X_{1} + \beta_{2}X_{2}. $$

We apply OLS estimation of coefficient and receive estimate of regression coeficeint. Let's do this in $\verb|R|$:

X1 <- runif(1000,0,1)
X2 <- runif(1000,1,2)
Y <- X1 + X2 + rnorm(1000,0,1)
model <- lm(Y ~ X1 + X2)
summary(model)

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)     0.1401     0.1828   0.766    0.444    
X1              1.0250     0.1150   8.909  < 2e-16 ***
X2              0.9247     0.1139   8.119 1.37e-15 ***
--- 
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.026 on 997 degrees of freedom
Multiple R-squared:  0.1276,    Adjusted R-squared:  0.1259 
F-statistic: 72.94 on 2 and 997 DF,  p-value: < 2.2e-16

Next, we want to test, if the regression coeficient $\beta_2$ is significant or not, so here is T-test for this:

$$H_{0}: \beta_{2} = 0$$ $$H_{1}: \beta_{2} \neq 0$$

Aproppiate test statistic is:

$$T_n = \frac{\hat{\beta_{2}}}{S.E(\hat{\beta_{2}})} \sim t_{n-r},$$

and p-value:$2min\{CDF_{t}(t_{0}), 1 - CDF_{t}(t_0)\}$, where $CDF_t$ is cummulative distribution function o $t$-distribution with $n-r$, in ou case $1000-3$ degrees of freedom. Let's suppose, that we also made test about submodel:

$$H_{0} Y \sim X_{1} \text{ holds} (M^{0})$$ $$H_{1} Y \sim X_{1} + X_{2} \text{ holds} (M).$$

Test statistics is $F = \frac{\frac{SSe^0 - SSe}{r - r_{0}}}{MSe} \sim F_{r-r_0,n-r}$ and p-value:$1-CDF_f(f_0)$, where $CDF_f$ is cummulative distribution function of F-distribution with $r-r_0$ and $n-r$ degrees of freedom. In $\verb|R|$ let's make an test about submodel:

m0 <- lm(Y ~ X1)
anova(m0,model)

Analysis of Variance Table

Model 1: Y ~ X1 
Model 2: Y ~ X1 + X2
    Res.Df    RSS Df Sum of Sq      F    Pr(>F)    
 1    998 1118.2                                  
 2    997 1048.8  1    69.352 65.926 1.373e-15 ***
 ---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Now there is a question, i know that if $X \sim t_{n-r}$, then $X^2 \sim F_{1,n-r}$. So if we squared t-value in our summary table, in line where regression coeficient $\hat{\beta}_2$ is, we geta f-value in output in test of submodel, so that's okay, but we have also same p-values, but p-value in summary table is computed from t-distribution, not squared distribution, and this numbers implies that $2min\{CDF_{t}(t_{0}), 1 - CDF_{t}(t_0)\} = 1-CDF_f(f_0)$, but i think this is not really true. What am i missing, how it is possible, that p-value which are computed from two different distributions are same ? (and this is not only for this case, but for every liner regression problems). Why p-values are same ? Please help.

In all cases i asuume normal linear model, that's why i assume those distribution of test statistics under $H_0$ hypothesis.

$\endgroup$
2
$\begingroup$

You should expect that the p-values are the same. If you reject the null hypothesis that $\beta_2$ = 0 (using the t-test), you should simultaneously reject the null hypothesis that the model without $\beta_2$ is adequate (using the F-test).

The p-value for the t-test is the probability that $|t_0| > t^{\alpha/2}_{n-r}$, and the p-value for the F-test is the probability that $F_0 > F^{\alpha/2}_{1, n-r}$. But note that these are mathematically identical statements, since squaring both sides of $|t_0| > t^{\alpha/2}_{n-r}$, gives you $F_0 > F^{\alpha/2}_{1, n-r}$, as you stated. Therefore, the p-values will always be numerically identical in this situation.

It's true that the p-value is computed from two different distributions, but they're found using different test statistics which are related to one another in a way that the p-value will always be identical.

$\endgroup$
3
  • $\begingroup$ this would be a good answer if you include how are the two test statistics related to each other and why will they always (or almost always?) produce the same p-value $\endgroup$
    – rep_ho
    Jan 8 '20 at 17:13
  • $\begingroup$ Thanks for your feedback @rep_ho. I updated my answer with some more details on how the statistics are related to one another. $\endgroup$ Jan 8 '20 at 18:14
  • $\begingroup$ Yes, this is right, thank you for your help. :) $\endgroup$
    – Bopinko
    Jan 9 '20 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.