# Exact equivalence of LR and Wald in linear regression under known error variance

Is it true that the LR statistic and the Wald statistic are numerically equivalent when testing a nested hypothesis in a linear regression when the error variance is known? Hence, is a squared t-statistic equal to the corresponding LR statistic?

Yes.

[Remark: I took the liberty to ask and answer this question in order to be able to supply the missing parts in this answer. As I believe the present question could be of independent interest I thought it might be helpful to state it separately. The exposition is based on the textbook by Paul Ruud.]

Consider a partitioned linear regression model $$y=X_1\beta_{01}+X_2\beta_{02}+\epsilon,$$ where $$y|X\sim\mathcal{N}(X\beta_0,\sigma^2I)$$.

We test $$$$H_0:\beta_{02}=0$$$$ The Wald test statistic is given by (see, e.g., here for the general formula) $$\begin{eqnarray*} \mathcal{W}&=&n\widehat{\beta}_2'\left[n\sigma^2\left[X_2'M_{X_1}X_2\right]^{-1}\right]^{-1}\widehat{\beta}_2\\ &=&\frac{\widehat{\beta}_2'X_2'M_{X_1}X_2\widehat{\beta}_2}{\sigma^2}\\ &=&\frac{y'M_{X_1}X_2(X_2'M_{X_1}X_2)^{-1}X_2'M_{X_1}X_2(X_2'M_{X_1}X_2)^{-1}X_2'M_{X_1}y}{\sigma^2}\\ &=&\frac{y'M_{X_1}X_2(X_2'M_{X_1}X_2)^{-1}X_2'M_{X_1}y}{\sigma^2}\\ &=&\frac{y'P_{M_{X_1}X_2}y}{\sigma^2}\\ &=:&\frac{y'P_{X_{2\bot1}}y}{\sigma^2}, \end{eqnarray*}$$ where the third equality follows from the Frisch-Waugh-Lovell theorem. Here, $$M_A$$ and $$P_A$$ denote the usual residual maker and projection matrices on $$A$$.

We now give an expression for the Likelihood ratio test statistic under known error variances.

Inserting the restricted and unrestricted estimators, denoted $$\widehat{\beta}$$ and $$\widehat{\beta}_R$$, into the sample log-likelihood yields, using $$\begin{eqnarray*} L(\widehat{\beta})&=&-\frac{n}{2}\log\left(2\pi\sigma^2\right)-\frac{(y-X\widehat{\beta})'(y-X\widehat{\beta})}{2\sigma^2} \end{eqnarray*}$$ and analogously for $$L(\widehat{\beta}_R)$$, the following expression for the $$\mathcal{L}\mathcal{R}$$-test statistic: $$\begin{eqnarray*}\mathcal{L}\mathcal{R}&=&2[L(\widehat{\theta})-L(\widehat{\theta}_R)]\\ &=&\frac{(y-X\widehat{\beta}_R)'(y-X\widehat{\beta}_R)-(y-X\widehat{\beta})'(y-X\widehat{\beta})}{\sigma^2}\\&=&\frac{y'(I-P_{X_1})y-y'(I-P_{X})y}{\sigma^2} \end{eqnarray*}$$

We now show that $$\mathcal{W}$$ may also be written in this format, $$\begin{eqnarray*} \mathcal{W}&=&\frac{y'(I-P_{X_1})y-y'(I-P_{X})y}{\sigma^2} \end{eqnarray*}$$ We first show that $$P_{X}=P_{X_1}+P_{X_{2\bot1}},$$ as a partition of $$X$$, $$X=(X_{A}\vdots X_{B}),$$ in orthogonal matrices $$X_{A}$$, $$X_{B}$$ ($$X_{A}'X_{B}=0$$) satisfies that $$P_{X}=P_{A}+P_{B}$$ This is so because $$\begin{eqnarray*} P_{X}&=&\left(\begin{array}{cc} X_{A} & X_{B} \\ \end{array} \right) \left( \begin{array}{cc} X_{A}'X_{A} & X_{A}'X_{B} \\ X_{B}'X_{A} & X_{B}'X_{B} \\ \end{array} \right)^{-1} \left( \begin{array}{c} X_{A}' \\ X_{B}' \\ \end{array} \right)\\ &=&\left(\begin{array}{cc} X_{A} & X_{B} \\ \end{array} \right) \left( \begin{array}{cc} X_{A}'X_{A} & 0 \\ 0 & X_{B}'X_{B} \\ \end{array} \right)^{-1} \left( \begin{array}{c} X_{A}' \\ X_{B}' \\ \end{array} \right)\\ &=&\left(\begin{array}{cc} X_{A} & X_{B} \\ \end{array} \right) \left( \begin{array}{cc} (X_{A}'X_{A})^{-1} & 0 \\ 0 & (X_{B}'X_{B})^{-1} \\ \end{array} \right) \left( \begin{array}{c} X_{A}' \\ X_{B}' \\ \end{array} \right)\\ &=& X_{A}(X_{A}'X_{A})^{-1}X_{A}'+X_{B}(X_{B}'X_{B})^{-1}X_{B}'\\ &=&P_{A}+P_{B} \end{eqnarray*}$$ We can apply this intermediate result to $$X_1$$ and $$X_{2\bot1}$$, as $$X_{2\bot1}'X_1=0$$. Hence, $$P_{X_{2\bot1}}=P_{X}-P_{X_1}.$$ Adding and subtracting $$y'Iy$$ in the numerator of the Wald statistic completes the proof.

It may be seen from, e.g., this answer that $$\mathcal{W}=t^2$$, with $$t$$ the t-ratio for some single coefficient, when we replace $$s^2$$ by $$\sigma^2$$, assumed known. When we need to estimate $$\sigma^2$$, it is however no longer true that $$\mathcal{W}=\mathcal{L}\mathcal{R}$$.

It may be established that the score test statistic is also numerically equivalent, which is skipped here for brevity.