# $t$-test and likelihood ratio test for testing the regression coefficient

I am studying hypothesis testing for the regression coefficient, it is given that

The hypotheses for testing the significance of any individual regression coefficient, such as $$\beta_{j},$$ are $$H_{0}: \beta_{j}=0, \quad H_{1}: \beta_{j} \neq 0$$ If $$H_{0}: \beta_{j}=0$$ is not rejected, then this indicates that the regressor $$x_{j}$$ can be deleted from the model. The test statistic for this hypothesis is $$t_{0}=\frac{\hat{\beta}_{j}}{\sqrt{\hat{\sigma}^{2} C_{j j}}}=\frac{\hat{\beta}_{j}}{\operatorname{se}\left(\hat{\beta}_{j}\right)}$$ where $$C_{j j}$$ is the diagonal element of $$\left(\mathbf{X}^{\prime} \mathbf{X}\right)^{-1}$$ corresponding to $$\hat{\beta}_{j} .$$ The null hypothesis $$H_{0}: \beta_{j}=0$$ is rejected if $$\left|t_{0}\right|>t_{\alpha / 2, n-k-1}$$.

1st question: It is just given that this $$t$$ is test statistics, but how to show/ prove that this hypothesis can be tested using the given $$t$$ - statistic.

Given the linear model $$G: \mathbf{Y}=\mathbf{X} \beta+\varepsilon,$$ where $$\mathbf{X}$$ is $$n \times p$$ of rank $$p$$ and $$\varepsilon \sim N_{n}\left(0, \sigma^{2} I_{n}\right),$$ we wish to test the hypothesis $$H: \mathbf{A} \beta=c,$$ where $$\mathbf{A}$$ is $$q \times p$$ of rank $$q$$.
The likelihood ratio test of $$H$$ is given by $$\Lambda=\frac{L\left(\hat{\beta}_{H}, \hat{\sigma}_{H}^{2}\right)}{L\left(\hat{\beta}, \hat{\sigma}^{2}\right)}=\left(\frac{\hat{\sigma}^{2}}{\hat{\sigma}_{H}^{2}}\right)^{n / 2}$$ and we can define $$F$$ statistic based on this ratio as $$F=\frac{n-p}{q}\left(\Lambda^{-2 / n}-1\right)$$ has an $$F_{q, n-p}$$ distribution when $$H$$ is true. We then reject $$H$$ when $$F$$ is too large.

2nd question: Can we prove that the likelihood ratio test is equivalent to the $$t$$-test in 1st question.

For linear regression a t-test and F-test are the same: Difference between t-test and ANOVA in linear regression

This is also true for your situation. You can transform the $$X$$ and $$\beta$$ in your equation $$Y=X\beta +\epsilon$$ such that $$\mathbf{A}\beta$$ is one of your parameters (*). And you can shift the Y-variable (subtract $$c\cdot \mathbf{A}\beta$$ from it) such that the test becomes $$\mathbf{A}\beta = 0$$. And then you have the same situation as typical regression where a t-test and F-test are the same.

This is a bit intuive explanation. Maybe it still needs the algebra to show that it actually works.

(*) Example of that transformation: recently we had a question Confidence interval for the difference of two fitted values from a linear regression model The question was about the distribution/confidence interval of the difference in the expectation of two points $$\hat{y}_1$$ and $$\hat{y}_2$$ (which can be expressed as some linear sum of the $$\beta$$). By a transformation of the regressors, this can be made directly equivalent to the distribution of a parameter in the regression.

• @IamKnull did you downvote this? Why not give a comment about it as well? I could improve the answer, but this blunt response is not very motivating. Commented Oct 20, 2020 at 9:31
1. This can be shown using the following results of a linear model $$\widehat{\epsilon} = y - X\widehat{\beta} \text{ and } \widehat{\epsilon} \sim N(0, \sigma^2(I-H))$$ $$\Rightarrow \left\| \widehat{\epsilon} \right\|^2 \sim \sigma^2 \chi^2_{n-p}$$ where p in the number of coefficients ($$\{\beta_0, \beta_1 ...\beta_{p-1} \}$$) $$\text{The above result comes from Eigen decomposition of I-H projection matrix}$$ Using the least squares estimates via orthogonalization of the jth column of matrix X, it can be shown that. $$\widehat{\beta}_j \sim N(\widehat{\beta}_j, \sigma^2 C_{jj})$$ and under the null hypothesis we have, $$\widehat{\beta}_j \sim N(0, \sigma^2 C_{jj})$$ or $$\frac{\widehat{\beta}_j}{\sigma\sqrt{ C_{jj}}} \sim N(0, 1)$$ Since we don't know the value of $$\sigma$$, we use the following test statistic $$t_j = \frac{\widehat{\beta}_j}{\widehat{\sigma}\sqrt{ C_{jj}}}$$ where $$\widehat{\sigma} = \frac{\widehat{\epsilon}}{n-p}$$ Now we can rewrite $$t_j$$ as $$t_j = \dfrac{\frac{\widehat{\beta}_j}{\sigma\sqrt{ C_{jj}}}}{\sqrt{\frac{\sigma^{-2} \left\| \widehat{\epsilon} \right\|^2}{n-p}}}$$ $$\Rightarrow t_j \sim \dfrac{N(0, 1)}{\sqrt{\dfrac{\chi_{n-p}^2}{n-p}}}$$ This distribution is called the t-distribution against which you can test your t-statistic.

2. Since F = $$t^2$$ you can use them interchangeably here.