# Prove Wald statistic = number of linear restrictions times F statistics

I'm considering $$F[J,n-k]= \displaystyle \frac{(e_{*}^{'}e_{*}-e'e) \backslash J}{e'e\backslash (n-k)}$$, where $$J$$ stands for number of restrictions. I want to prove $$W=(Rb-q)'(Rs^2(X'X)^{-1}R')^{-1}(Rb-q)=JF$$, where $$s^2=\displaystyle{\frac{e'e}{n-k}}$$. So it is enough to show $$(e_{*}^{'}e_{*}-e'e)$$ is $$(Rb-q)'(R(X'X)^{-1}R')^{-1}(Rb-q)$$,$$R$$ stands for the restrictions.

In Greene's textbook, he states this, but I don't know how to derive it.

In a particular, probably simpler example, consider a partitioned linear regression model $$Y=X_1\beta_1+X_2\beta_2+\epsilon$$, and $$H_0$$:$$\beta_2=0$$, so $$Rb-q=b_2$$ which is an OLS estimator of $$\beta_2$$.$$\beta_1$$ is $$k_1\times 1$$ and $$\beta_2$$ is $$k_2\times 1$$ so $$J$$ should be $$k_2$$.We want $$k_2F=W$$. But I don't know how to derive it even in this example.

Could you please give a proof in example and then maybe in general case?

• Possibly helpfull is the relationship between t-statistic and F-statistic. Relationship between the t-statistic and F-statistic in simple linear regression Commented Apr 29 at 5:39
• It sounds to me as if you are looking for the proof which Green shows, so what exactly is your question? Btw, could you also give the edition of Green, the page and formula number.
– BenP
Commented Apr 30 at 7:14
• I mean, you say "I want to proof W= ..... = JF". Which is what Green proofs.
– BenP
Commented Apr 30 at 7:27
• Now I think I see what you mean. Define the F in terms of the residuals of the models with and without restrictions? Maybe you should explain that in the question more clearly.
– BenP
Commented Apr 30 at 9:33
• The formulation of your question is not very clear. Could you maybe change it into something like: "How to use the difference in residual sums of squares of two nested linear models to obtain the usual F statistic?".
– BenP
Commented Apr 30 at 19:23

I assume that your main question is about how the residuals $$e$$ of the full and those of the restricted model $$e_*$$ can be used to derive the F statistic. The F statistics is used to test the null hypothesis that the (linear) restrictions applied to a set of regression coefficients hold true (for the population, the data generatig proces).

In your question you refer to the book of Green, which therefore I will do as well. The notation used in the answer conforms to Green's notation.

We have a linear regression model and we want to test some linear restriction on the regression coefficients, like $$b_1=0$$ or $$b_1=b_2=0$$ or $$b_1=b_4=3$$ or $$b_1+b_2-b_5=2$$, just to mention a few linear restrictions that could be tested.

We have a full model, no restrictions applied, the regression coefficients denoted by $$b$$. Also, we have a restricted model, the regression coefficients denoted by $$b_*$$. The residuals of the full model are denoted as $$e$$ and those of the restricted model as $$e_*$$. We'd like to use the difference of the sums of squares of $$e$$ and $$e_*$$ to obtain the F statistic. In vector notation, we are interested in using $$e_*'e_* - e'e$$ to obtain the value of the F statistic under $$H_0$$.

Green shows that the difference $$b_*-b= -Cm$$. , where $$m = Rb-q$$. $$R$$ is a vector or matrix specifying the restrictions (see Green) and $$q$$ is a vector containing values for the specified restrictions (also see Green). $$C$$ is a matrix:

$$C = (X\,'X)^{-1}R\,'P$$ where $$P=[R\,(X'X)^{-1}R\,']^{-1}$$. Also:

$$C'= PR(X'X)^{-1}$$, because both $$P$$ and $$(X'X)^{-1}$$ are symmetric.

The above formula for $$C$$ is arrived at by applying what Green describes in the paragraph "restriced least squares estimation". It's a pitty that the equation for $$b_*-b= -Cm$$ is not explained (or at least referred to) in the later paragraph where Green develops the F statistic in terms of the difference $$e_*'e_* - e'e$$ which we focus on in this question and answer.

Because $$b_*= b-Cm$$ it holds that $$Y-Xb_*=Y-Xb-XCm$$. Hence:

$$e_*\,'e_* = (Y-Xb_*)\,'(Y-Xb_*) = (Y-Xb+XCm)\,'(Y-Xb+XCm)=$$
$$(e+XCm)\,'(e+XCm) = (e'+ m'C'X')(e+XCm)$$

This can be elaborated further as:

$$e'e+m'C'X'e \quad + \quad e'XCm + m'C'X'XCm = e'e + 0 + 0 + m'C'X'XCm$$

And so we obtain:

$$e_*'e_* - e'e = m'C'X'XCm$$

Focusing now and $$C'X'XC$$ and substituting the expression for C given above, we get:

$$C'X'XC = [PR(X'X)^{-1}]\,\, X'X \,\, [(X'X)^{-1}R\,'P]=PR(X'X)^{-1}R\,'P = PP^{-1}P=P$$

Finally we get that:

$$e_*'e_* - e'e = m'C'X'XCm = m'Pm=m'[R'(X'X)^{-1}R']^{-1}m$$

or:

$$e_*'e_* - e'e = (Rb-q)'\, [R'(X'X)^{-1}R']^{-1}\,(Rb-q)$$

This final expression can be used to get the F statistic mentioned at the beginnig of this question.

• @Chang Henry is this indeed the answer you were looking for? I'm stil not sure. If not, let me know.
– BenP
Commented May 2 at 7:05