# Proof that Sargan test statistic is distributed $\chi^2$

Assume that

$$\textbf{y} = X\boldsymbol\beta + \boldsymbol\varepsilon$$

, where $$\boldsymbol\varepsilon$$ is an $$n \times 1$$ random vector and that X is an $$n \times k$$ matrix of regressors; $$m$$ of these regressors are endogenous. Let $$Z$$ be an $$n \times (k-m+l)$$ matrix where the endogenous regressors are replaced by exogenous instruments. If $$\boldsymbol\beta$$ is estimated by 2SLS

$$\hat{\boldsymbol\beta}_{2SLS} = \left(X^\prime P_Z X \right)^{-1} X^\prime P_Z \textbf{y}$$ $$P_Z= Z\left(Z^\prime Z \right)^{-1} Z^\prime$$

, it is (supposedly) possible to show that if $$E[\boldsymbol{z}_i^\prime\varepsilon_i]=\textbf{0}$$, then

$$n\frac{\hat{\boldsymbol\varepsilon}^\prime P_Z \hat{\boldsymbol\varepsilon}}{\hat{\boldsymbol\varepsilon}^\prime \hat{\boldsymbol\varepsilon}} \sim \chi^2 \left(l-m \right)$$

See, e.g., https://en.wikipedia.org/wiki/Sargan%E2%80%93Hansen_test . I have not been able to find an explanation of how it is possible to show this. That is, I have not found one that I am able to understand; see, e.g., http://www.jstor.org/stable/1907619 for the original article. My guess is that we would invoke the Central Limit Theorem

$$\frac{1}{\sqrt n} Z^\prime \hat{\boldsymbol\varepsilon} \sim N(0,\Omega)$$

and assume that $$\Omega$$ is invertible, such that

$$\frac{1}{\sqrt n} \Omega^{-\frac{1}{2}} Z^\prime \hat{\boldsymbol\varepsilon} \sim N(0,I)$$

Then

$$\left(\frac{1}{\sqrt n} \Omega^{-\frac{1}{2}} Z^\prime \hat{\boldsymbol\varepsilon} \right)^\prime \left(\frac{1}{\sqrt n} \Omega^{-\frac{1}{2}} Z^\prime \hat{\boldsymbol\varepsilon} \right) \sim \chi^2 (l-m)$$

, since it is a sum of independent squares. Assuming homoskedasticity, we can replace $$\Omega$$ by

$$\hat{\Omega} = \frac{\sigma^2_{\hat{\varepsilon}}}{n} Z^\prime Z$$

, which yields the Sargan test statistic.

My question is if this proof is correct? Also, how can we show that the degrees of freedom are $$l-m$$? The $$l$$-part makes sense, since all except $$l$$ elements of $$Z^\prime \hat{\boldsymbol\varepsilon}$$ are 0 by construction. Any help would be much appreciated.

• What do you mean by saying "since all except $l$ elements of $Z′\hat{\epsilon}$ are 0 by construction"? There are just $l$ elements in $Z′\hat{\epsilon}$. Oct 31, 2019 at 14:22
• There are $k-m+l$ elements in $Z^\prime \hat{\boldsymbol\varepsilon}$. For the exogenous variables that are not excluded from the second stage, $x_1, ..., x_{k-m}$, we have $\sum_{i=1}^n x_{ji }\hat{\varepsilon}_i = 0$ by construction. Nov 1, 2019 at 12:10
• OK, I did not read your matrix dimensions right. Nov 3, 2019 at 16:16

I will give a proof for a general GMM test statistic of overidentifying restrictions, which evaluates the GMM criterion function at the GMM estimate.

The Sargan statistic is a special case of this statistic under conditional homoskedasticity.

The notation and exposition follow Hayashi (2000) Econometrics, so that, unlike in your question, instruments are denoted by $$X$$ (there are $$K$$ of them) and regressors by $$Z$$ (there are $$L$$ of them). $$S_{xz}$$ etc. are then corresponding sample moment matrices. $$\widehat{\delta}_{GMM}$$ is the GMM estimate of the true coefficient $$\delta$$. $$\widehat{S}^{-1}$$ is the efficient weighting matrix. $$g_i(\tilde{\delta}):=x_{i}(y_i-z_i'\tilde{\delta})$$ and $$g_n(\tilde{\delta})=\frac{1}{n}\sum_{i=1}^{n}g_{i}(\tilde{\delta})$$

$$\begin{equation*} J\bigl(\widehat{\delta}_{GMM},\widehat{S}^{-1}\bigr)\to_d\chi^{2}(K-L). \end{equation*}$$

Proof:

First, note that $$\begin{eqnarray*} g_n(\widehat{\delta}_{GMM})&=&s_{xy}-S_{xz}\widehat{\delta}_{GMM}\\ &=&s_{xy}-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1}s_{xy}\\ &=&\bigl(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1}\bigr)s_{xy}\\ &=:&\widehat{B}s_{xy} \end{eqnarray*}$$ Further, $$\begin{eqnarray} \widehat{B}s_{xy}&=&(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1})s_{xy}\notag\\ &=&(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1})(S_{xz}\delta+g_n(\delta))\notag\\ &=&\underbrace{(S_{xz}-S_{xz}\underbrace{(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1}S_{xz}}_{I})}_{0}\delta+\widehat{B}g_n(\delta)\notag\\ &=&\widehat{B}g_n(\delta)\label{bgndelta} \end{eqnarray}$$

Using the decomposition $$\widehat{S}^{-1}=C'C$$ we obtain $$\widehat{B}'\widehat{S}^{-1}\widehat{B}=(C\widehat{B})'(C\widehat{B})$$. Now, $$\begin{eqnarray*} C\widehat{B}&=&C(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1})\\ &=&C-CS_{xz}(S_{xz}'C'CS_{xz})^{-1}S_{xz}'C'C\\ &=&C-A(A'A)^{-1}A'C\\ &=&(I-A(A'A)^{-1}A')C\\&=:&MC, \end{eqnarray*}$$ where $$A:=CS_{xz}$$ and, as usual, $$M$$ is symmetric and idempotent, so that $$\begin{equation}\label{Jtestproof3} \widehat{B}'\widehat{S}^{-1}\widehat{B}=C'M'MC=C'MC. \end{equation}$$ Then, $$\begin{eqnarray*} tr(M)&=&tr(I-A(A'A)^{-1}A')\\&=&K-tr((A'A)^{-1}A'A)\\ &=&K-tr(I_L)\\&=&K-L. \end{eqnarray*}$$ Now, take $$D'D=S^{-1}$$ so that $$C\to_pD$$.

With the baseline assumption that, at true value, the sample moment conditions satisfying a CLT, $$\sqrt{n}g_n(\delta)\to_dN(0,S)$$ it holds that $$\begin{eqnarray} C\sqrt{n}g_n(\delta)&\to_d&N(0,DSD')\notag\\&=&N(0,D(D'D)^{-1}D')\notag \\&=&N(0,DD^{-1}D'^{-1}D')\notag \\&=&N(0,I)\label{hansenovern} \end{eqnarray}$$ Hence, $$\begin{eqnarray} J\bigl(\widehat{\delta}_{GMM},\widehat{S}^{-1}\bigr)&=&n\cdot g_n(\widehat{\delta}_{GMM})'\widehat{S}^{-1}g_n(\widehat{\delta}_{GMM})\notag\\ &=&n\cdot g_n(\delta)'\widehat{B}'\widehat{S}^{-1}\widehat{B}g_n(\delta)\notag\\ &=&n\cdot g_n(\delta)'C'MCg_n(\delta)\notag\\ &=&\sqrt{n}\cdot(Cg_n(\delta))'M\sqrt{n}Cg_n(\delta)\label{hansenoveridasy} \end{eqnarray}$$ Thus, this is asymptotically is a quadratic form in normally distributed vectors and an idempotent matrix $$M$$ with $$tr(M)=rk(M)=K-L$$. Thus, (see e.g. Thm. A.87 in Toutenburg and Rao (1999) Linear Models: Least Squares and Alternatives) $$J\bigl(\widehat{\delta}_{GMM},\widehat{S}^{-1}\bigr)\to_d\chi^{2}(K-L)$$

• Cheers! I will work through your proof when I get the time. The reference was helpful, found it here. I'm not sure that it answers my question, though. I'm not saying that it doesn't, either. I'm just not sure yet... Nov 1, 2019 at 12:57

I will use your notation and write $$M_A = Id - P_A = Id - A^T (A^T A)^{-1} A$$ for the projection onto the orthogonal complement of $$A$$. Let's assume all k regressors in $$X$$ are endogenous, i.e., $$k=m$$. Otherwise, replace $$Z, X, y$$ by their projection onto the orthogonal complement of the exogenous regressors. Thus, $$\dim(X) = k \leq \dim(Z) = l$$.

The J-statistic is

\begin{align*} J &= \frac{1}{\hat \sigma_\varepsilon^2} (y - X \hat\beta)^T P_Z(y - X\hat\beta). \end{align*}

Expand

\begin{align*} P_Z(y - X \hat\beta) &= P_Z\left(X \beta_0 + \varepsilon - X (X^T P_Z X)^{-1} X^T P_Z (X\beta_0 + \varepsilon)\right) \\ &= P_Z \varepsilon - P_Z X (X^T P_Z X)^{-1}X^T P_Z \varepsilon \\ &= P_Z M_{P_Z X} \varepsilon = P_{M_X Z} \varepsilon \end{align*}

Thus $$J = \frac{1}{\hat\sigma_\varepsilon^2} \varepsilon^T P_{M_X Z} \varepsilon.$$

As $$rank(P_{M_X Z}) = l - k$$, the result $$J \overset{d}{\to} \chi^2(l - k)$$ follows from a CLT argument. If the $$\varepsilon$$ are normal, then $$\frac{1}{l - k} J \sim F_{l - k, n - l}$$.