Proof that Sargan test statistic is distributed $\chi^2$

Question

Assume that

$$\textbf{y} = X\boldsymbol\beta + \boldsymbol\varepsilon $$

, where $\boldsymbol\varepsilon$ is an $n \times 1$ random vector and that X is an $n \times k$ matrix of regressors; $m$ of these regressors are endogenous. Let $Z$ be an $n \times (k-m+l)$ matrix where the endogenous regressors are replaced by exogenous instruments. If $\boldsymbol\beta$ is estimated by 2SLS

$$ \hat{\boldsymbol\beta}_{2SLS} = \left(X^\prime P_Z X \right)^{-1} X^\prime P_Z \textbf{y}$$ $$ P_Z= Z\left(Z^\prime Z \right)^{-1} Z^\prime $$

, it is (supposedly) possible to show that if $E[\boldsymbol{z}_i^\prime\varepsilon_i]=\textbf{0} $, then

$$ n\frac{\hat{\boldsymbol\varepsilon}^\prime P_Z \hat{\boldsymbol\varepsilon}}{\hat{\boldsymbol\varepsilon}^\prime \hat{\boldsymbol\varepsilon}} \sim \chi^2 \left(l-m \right)$$

See, e.g., https://en.wikipedia.org/wiki/Sargan%E2%80%93Hansen_test . I have not been able to find an explanation of how it is possible to show this. That is, I have not found one that I am able to understand; see, e.g., http://www.jstor.org/stable/1907619 for the original article. My guess is that we would invoke the Central Limit Theorem

$$ \frac{1}{\sqrt n} Z^\prime \hat{\boldsymbol\varepsilon} \sim N(0,\Omega) $$

and assume that $\Omega$ is invertible, such that

$$ \frac{1}{\sqrt n} \Omega^{-\frac{1}{2}} Z^\prime \hat{\boldsymbol\varepsilon} \sim N(0,I) $$

Then

$$ \left(\frac{1}{\sqrt n} \Omega^{-\frac{1}{2}} Z^\prime \hat{\boldsymbol\varepsilon} \right)^\prime \left(\frac{1}{\sqrt n} \Omega^{-\frac{1}{2}} Z^\prime \hat{\boldsymbol\varepsilon} \right) \sim \chi^2 (l-m) $$

, since it is a sum of independent squares. Assuming homoskedasticity, we can replace $ \Omega $ by

$$ \hat{\Omega} = \frac{\sigma^2_{\hat{\varepsilon}}}{n} Z^\prime Z$$

, which yields the Sargan test statistic.

My question is if this proof is correct? Also, how can we show that the degrees of freedom are $l-m$? The $l$-part makes sense, since all except $l$ elements of $Z^\prime \hat{\boldsymbol\varepsilon} $ are 0 by construction. Any help would be much appreciated.

Answer 1

I will give a proof for a general GMM test statistic of overidentifying restrictions, which evaluates the GMM criterion function at the GMM estimate.

The Sargan statistic is a special case of this statistic under conditional homoskedasticity.

The notation and exposition follow Hayashi (2000) Econometrics, so that, unlike in your question, instruments are denoted by $X$ (there are $K$ of them) and regressors by $Z$ (there are $L$ of them). $S_{xz}$ etc. are then corresponding sample moment matrices. $\widehat{\delta}_{GMM}$ is the GMM estimate of the true coefficient $\delta$. $\widehat{S}^{-1}$ is the efficient weighting matrix. $g_i(\tilde{\delta}):=x_{i}(y_i-z_i'\tilde{\delta})$ and $g_n(\tilde{\delta})=\frac{1}{n}\sum_{i=1}^{n}g_{i}(\tilde{\delta})$

\begin{equation*} J\bigl(\widehat{\delta}_{GMM},\widehat{S}^{-1}\bigr)\to_d\chi^{2}(K-L). \end{equation*}

Proof:

First, note that \begin{eqnarray*} g_n(\widehat{\delta}_{GMM})&=&s_{xy}-S_{xz}\widehat{\delta}_{GMM}\\ &=&s_{xy}-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1}s_{xy}\\ &=&\bigl(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1}\bigr)s_{xy}\\ &=:&\widehat{B}s_{xy} \end{eqnarray*} Further, \begin{eqnarray} \widehat{B}s_{xy}&=&(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1})s_{xy}\notag\\ &=&(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1})(S_{xz}\delta+g_n(\delta))\notag\\ &=&\underbrace{(S_{xz}-S_{xz}\underbrace{(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1}S_{xz}}_{I})}_{0}\delta+\widehat{B}g_n(\delta)\notag\\ &=&\widehat{B}g_n(\delta)\label{bgndelta} \end{eqnarray}

Using the decomposition $\widehat{S}^{-1}=C'C$ we obtain $\widehat{B}'\widehat{S}^{-1}\widehat{B}=(C\widehat{B})'(C\widehat{B})$. Now, \begin{eqnarray*} C\widehat{B}&=&C(I-S_{xz}(S_{xz}'\widehat{S}^{-1}S_{xz})^{-1}S_{xz}'\widehat{S}^{-1})\\ &=&C-CS_{xz}(S_{xz}'C'CS_{xz})^{-1}S_{xz}'C'C\\ &=&C-A(A'A)^{-1}A'C\\ &=&(I-A(A'A)^{-1}A')C\\&=:&MC, \end{eqnarray*} where $A:=CS_{xz}$ and, as usual, $M$ is symmetric and idempotent, so that \begin{equation}\label{Jtestproof3} \widehat{B}'\widehat{S}^{-1}\widehat{B}=C'M'MC=C'MC. \end{equation} Then, \begin{eqnarray*} tr(M)&=&tr(I-A(A'A)^{-1}A')\\&=&K-tr((A'A)^{-1}A'A)\\ &=&K-tr(I_L)\\&=&K-L. \end{eqnarray*} Now, take $D'D=S^{-1}$ so that $C\to_pD$.

With the baseline assumption that, at true value, the sample moment conditions satisfying a CLT, $\sqrt{n}g_n(\delta)\to_dN(0,S)$ it holds that \begin{eqnarray} C\sqrt{n}g_n(\delta)&\to_d&N(0,DSD')\notag\\&=&N(0,D(D'D)^{-1}D')\notag \\&=&N(0,DD^{-1}D'^{-1}D')\notag \\&=&N(0,I)\label{hansenovern} \end{eqnarray} Hence, \begin{eqnarray} J\bigl(\widehat{\delta}_{GMM},\widehat{S}^{-1}\bigr)&=&n\cdot g_n(\widehat{\delta}_{GMM})'\widehat{S}^{-1}g_n(\widehat{\delta}_{GMM})\notag\\ &=&n\cdot g_n(\delta)'\widehat{B}'\widehat{S}^{-1}\widehat{B}g_n(\delta)\notag\\ &=&n\cdot g_n(\delta)'C'MCg_n(\delta)\notag\\ &=&\sqrt{n}\cdot(Cg_n(\delta))'M\sqrt{n}Cg_n(\delta)\label{hansenoveridasy} \end{eqnarray} Thus, this is asymptotically is a quadratic form in normally distributed vectors and an idempotent matrix $M$ with $tr(M)=rk(M)=K-L$. Thus, (see e.g. Thm. A.87 in Toutenburg and Rao (1999) Linear Models: Least Squares and Alternatives) $$J\bigl(\widehat{\delta}_{GMM},\widehat{S}^{-1}\bigr)\to_d\chi^{2}(K-L) $$

Answer 2

I will use your notation and write $M_A = Id - P_A = Id - A^T (A^T A)^{-1} A$ for the projection onto the orthogonal complement of $A$. Let's assume all k regressors in $X$ are endogenous, i.e., $k=m$. Otherwise, replace $Z, X, y$ by their projection onto the orthogonal complement of the exogenous regressors. Thus, $\dim(X) = k \leq \dim(Z) = l$.

The J-statistic is

\begin{align*} J &= \frac{1}{\hat \sigma_\varepsilon^2} (y - X \hat\beta)^T P_Z(y - X\hat\beta). \end{align*}

Expand

\begin{align*} P_Z(y - X \hat\beta) &= P_Z\left(X \beta_0 + \varepsilon - X (X^T P_Z X)^{-1} X^T P_Z (X\beta_0 + \varepsilon)\right) \\ &= P_Z \varepsilon - P_Z X (X^T P_Z X)^{-1}X^T P_Z \varepsilon \\ &= P_Z M_{P_Z X} \varepsilon = P_{M_X Z} \varepsilon \end{align*}

Thus $$ J = \frac{1}{\hat\sigma_\varepsilon^2} \varepsilon^T P_{M_X Z} \varepsilon. $$

As $rank(P_{M_X Z}) = l - k$, the result $J \overset{d}{\to} \chi^2(l - k)$ follows from a CLT argument. If the $\varepsilon$ are normal, then $\frac{1}{l - k} J \sim F_{l - k, n - l}$.