Asymptotic Distribution of the Wald Test Statistic

Question

I am trying to understand the asymptotic distribution of the Wald test statistic, specifically under the alternative hypothesis which I've found little reference to.

For clarity, the binary hypothesis for an unknown parameter vector $\theta$ of size $(r \times 1)$ is:

\begin{equation} \text{H}_0 : \theta = \theta_0 \nonumber \\ \text{H}_1 : \theta \neq \theta_0 \end{equation}

and the resulting log likelihood ratio, when assuming the asymptotic PDF of the MLE is attained ($N \rightarrow \infty$), can be modified to yield the Wald Test Statistic:

\begin{equation} 2 \ln L_G({\bf x} ) = T_W({\bf x}) = (\hat{\theta}_1 - \theta_0)^T {\bf I}(\hat{\theta}_1) (\hat{\theta}_1 - \theta_0) \end{equation}

where $\hat{\theta}_1$ denotes the unrestricted MLE and ${\bf I}$ is the FIM. Now, as we seek to attain the asymptotic distribution of this test statistic we can note:

$ \hat{\theta}_1 \sim \begin{cases} \hfill \mathcal{N}\Big( \theta_0, {\bf I}^{-1}(\theta_0) \Big) \hfill & \text{under } \text{H}_0 \\ \hfill \mathcal{N}\Big( \theta_1, {\bf I}^{-1}(\theta_1) \Big) \hfill & \text{under } \text{H}_1\\ \end{cases} $

So that as $N \rightarrow \infty$, under the null hypothesis ($\text{H}_0$) :

\begin{equation} T_W({\bf x}) = (\hat{\theta}_1 - \theta_0)^T {\bf I}(\theta_0) (\hat{\theta}_1 - \theta_0) \sim \chi^2_r \end{equation}

But it is the distribution under $\text{H}_1$ I get confused. A variety of texts/proofs (i.e. Kay Vol II) say the following, as $N \rightarrow \infty$:

\begin{equation} {\bf I}(\hat{\theta}_1) (\hat{\theta}_1 - \theta_0) = {\bf I}(\theta_0) (\hat{\theta}_1 - \theta_0) \end{equation}

The above equation I have no clue how to validate. Anyways, the proof continues so that under $\text{H}_1$:

\begin{equation} T_W({\bf x}) = (\hat{\theta}_1 - \theta_0)^T {\bf I}(\theta_1) (\hat{\theta}_1 - \theta_0) \sim \chi^2_r (\lambda) \end{equation} where the non centrality parameter is equal to: \begin{equation} \lambda = ( \theta_1 - \theta_0 )^T{\bf I}(\theta_1) ( \theta_1 - \theta_0 ) \end{equation}

or equivalently, as $N \rightarrow \infty$: \begin{equation} \lambda = ( \theta_1 - \theta_0 )^T{\bf I}(\theta_0) ( \theta_1 - \theta_0 ) \end{equation}

How can this possibly be true? Unless there's a strong assumption taken here that I am unaware of, there is no reason, even asymptotically, the variances of these two values ($\theta_1$ and $\theta_0$) should be equal. What am I missing? Something inherent in the Wald Test Statistic?

Any input on this would be appreciated as I'm completely vexed. Thanks!

Answer 1

I believe I found the answer. What I suspected is actually true.

\begin{equation} {\bf I}(\theta_1) \approx {\bf I}(\theta_0) \end{equation}

Because it is assumed $\theta_1$ is near, or "contingent" to $\theta_0$. And as $N \rightarrow \infty$ we are guaranteed the convergence of the MLE to these true values. As such, this assumption is "validated." It is highly limiting, but makes sense why the distribution under the alternative hypothesis is rarely mentioned in literature.

This also makes sense since the Wald, Rao, and likelihood tests are asymptotically equal up to a 1st order. From 2nd order and up divergences happen.

Answer 2

A slight generalisation of @pellis's answer.

Given a sequence of distributions $P_n\stackrel{d}{\to}P$, a sequence $Q_n\stackrel{d}{\to} Q$ is mutually contiguous if a sequence of events $A_n$ has $P_n(A_n)\to 0$ iff it has $Q_n(A_n)\to 0$. This is relevant to testing if you think of $A_n$ as "does not reject the null". For non-contiguous sequences of alternatives the power just goes to 1. If you're interested in anything detailed about power, the contiguous asymptotics are often going to be the relevant ones.

For contiguous sequences of alternatives very generally, a standard Normal test statistic gets a location shift to a mean depending on how much the distribution changes (in Kullback-Leibler terms) and whether it's in a direction that affects the parameter you're testing. This is called LeCam's Third Lemma. There are simpler versions of it for correctly specified parametric models, which basically say the change in the information when you change $\theta$ is zero to first order in the size of the change in $\theta$.

Roughly, $$\frac{\partial}{\partial\theta}\left(\frac{\partial\ell}{\partial\theta}\right)^2 =2\left(\frac{\partial^2\ell}{\partial\theta^2}\right)\left(\frac{\partial\ell}{\partial\theta}\right) $$ and at $\theta_0$ in large samples $\partial\ell/\partial\theta$ is close to zero and $\partial^2\ell/\partial\theta^2$ is close to $-I(\theta_0)$