1
$\begingroup$

I've read about two ways to test the significance of just one predictor when fitting a linear model with least squares.

Suppose I fit the following linear model using least squares using standard assumptions. $$y = X \beta + \varepsilon \tag{1}$$ where $\beta = (\beta_1, \cdots, \beta_i, \cdots, \beta_p)^T$.

The first method I've seen is the most basic way, where we compute the test statistic $$t = \frac{\hat{\beta}_i}{se(\hat{\beta}_i)} . \tag{2}$$ Then assuming that we've already shown that $\hat{\beta}_i \sim N(X\beta, \sigma^2 I)$ under the assumption that $X$ is non-stochastic, the test statistic $t$ follows a $T$ distribution with $\nu$ degrees of freedom, $$t \sim \frac{Z}{\sqrt{\chi^2/\nu}} = T_\nu .$$

The second approach I've seen is a likelihood ratio testing approach, where $\Omega$ is a linear model with $p$ parameters and $\omega$ is a linear model with $q$ parameters formed by imposing linear constraints on the model $\Omega$. The likelihood ratio test then gives us the following test statistic with an F distribution. $$f_{general} = \frac{(RSS_\omega - RSS_\Omega)/(p-q)}{RSS_\Omega / (n-p)} \sim F_{p-q,n-p}$$.

If we let $\Omega$ be the model in Equation (1), and let $\omega$ be that model but with $\beta_i$ removed (or imposing the constraint that $\beta_i=0$) for some particular $i$, then we get $$f = \frac{(RSS_\omega - RSS_\Omega)}{RSS_\Omega / (n-p)} \sim F_{1,n-p} . \tag{3}$$

We know that the square of a T-distributed random variable is an F-distribution random variable, $$T^2_\nu = F_{1,\nu}.$$

So what's the difference between testing with the test statistic in Equation (2) versus with the test statistic in Equation (3)?

I think that the Neyman-Pearson Lemma tells us that testing with Equation (3) would give us the most powerful test for $H_0: \beta_i=0$, but can the same be said of Equation (2)?

Also, I don't see why it can't be that $t^2 \neq f$, so is it possible for (2) to fail to reject $H_0$ but then for (3) to actually reject $H_0$ for the same model and data?

$\endgroup$
1
$\begingroup$

(2) and (3) are equivalent. They Reject or not at the same time. And yes the value of the t statistic squared will be the value of the F stat (because we have 1 numerator df). All you are doing is comparing the t and F statistics to different Tables where the alpha % tail value of the t table is squared to give the corresponding value on the F table. Look at the tabled values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.