4
$\begingroup$

As far as I know, I have two options for tests in linear regression. The $F$-test for the model (if it explains more variance than it has error variance) and the $t$-test (to see if the slope is not zero). With more than one predictor, I can see why both tests are there. But in my case, I only have one predictor. In my opinion, the $F$-test and the $t$-test do the same in this case because if the slope is zero, it is exactly the model it is compared to in the $F$-test.

Where is the flaw in my logic?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Technically, in OLS you can also test H$_{0}\text{: }r=0$ (where $r=\sqrt{R^{2}}$, or, if your prefer to be pedantic $r = \text{sign}(\beta)\sqrt{R^{2}}$), and you can also go wild with a test of H$_{0}\text{: }\alpha=0$. Just sayin'... $\endgroup$
    – Alexis
    May 14, 2014 at 17:54

2 Answers 2

Reset to default
5
$\begingroup$

No flaw. If your model is $y=\alpha + \beta x + \varepsilon$, your t-statistic is distributed as $t_{n-2}$, where $n$ is the number of observations and $2$ the number of parameters (intercept and slope), your F-statistic is distributed as a $F_{1,n-2}$.

In general, if $X\sim t_{n}$, then $X^2\sim F_{1,n}$. This if why you can see that the F-statistic is equal to the square t-statistics (the one for the slope) and their p-values are equal. In this sense the t-test and the F-test "do the same".

Improve this answer
$\endgroup$
2
$\begingroup$

The $F$-test that comes with a simple linear regression does not test both the slope and the intercept. It tests only the slope. The $p$-values from the $F$-test and the $t$-test of the slope of your predictor variable will be the same. Consider this simple example in R:

set.seed(8)  # this makes the example perfectly reproducible
x = rnorm(20, mean=0, sd=1)
y = rnorm(20, mean=5, sd=1)
model = lm(y~x)  # this fits a simple linear regression model
summary(model)
# ...
# Coefficients:
#             Estimate Std. Error t value Pr(>|t|)    
# (Intercept) 5.169948   0.265024  19.507 1.48e-13 ***
# x           0.002842   0.262160   0.011    0.991    
# ...
# Residual standard error: 1.162 on 18 degrees of freedom
# Multiple R-squared:  6.53e-06,    Adjusted R-squared:  -0.05555 
# F-statistic: 0.0001175 on 1 and 18 DF,  p-value: 0.9915

The two $p$-values are the same, despite the fact that the intercept is wildly significant.

In addition, it is not correct to say that the $F$-test is checking "if [the model] explains more variance than it has error variance". There are several valid ways to understand what the $F$-test assesses, but in this context perhaps it is better thought of as checking if the addition of the predictor variable provides a significant improvement in model fit over the null model. With only one variable, and no other model specified, the null model is the intercept only model. That is why they are both tests of the same hypothesis and yield the same $p$-value.

To provide a more direct answer to your question: I don't see any flaw in your logic. However, it isn't very meaningful to say that you have two options for testing your model. Although they take different routes, they get to exactly the same place. Moreover, the $F$-statistic in this case will always be equal to $t^2$, and the $F$-test's denominator degrees of freedom will always be equal to the $t$-test's degrees of freedom.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.