Justification for use of $\chi^2(1)$ in Wald and score test

Question

In a recent exam, we were asked to justify the use of the $\chi^2(1)$ distribution in performing the Wald or Rao's score test. There was only 1 mark for this (approx. 2.5 mins worth of time). My answer was

The Wald and score test statistics are based on various approximations to the log-likelihood ratio, which are valid and equivalent in large samples when $H_0$ is true. For example, the approximation for the 2nd Wald statistic is

$$ 2\log(LR)\simeq (\hat{\theta}_n-\theta_0)^2E\{-\ell''(\theta)\}|_{\theta_0} =(\hat{\theta}_n-\theta_0)^2I(\theta_0) $$

where $I(\theta_0)$ is the Fisher information at $\theta_0$. Then, using asymptotic normality,

$$\hat{\theta}_n\approx N \left ( \theta_0,\frac{1}{ni(\theta_0)}\right )=N \left ( \theta_0,\frac{1}{I(\theta_0)}\right )$$

which yields $(\hat{\theta}_n-\theta_0)\sqrt{I(\theta_0)} \approx N(0,1)$

Since $2\log(LR)$ is approximately equal to the square of the LHS of this when $n$ is large, it is approximately distributed as the square of a standard normal random variable, that is, as $\chi^2(1)$

The marker wrote "insufficient" and I got zero for this. As it is a summative exam, they won't give any feedback, nor engage in any discussion about it. I was wondering if anyone here can explain what I have missed or where I went wrong. I'm not very good with latex so I hope I didn't make any mistakes in the typing !

This is for an elective module in statistical theory in the final year of an undergraduate maths degree. Thanks !

Edit: There is a formal procedure to have my script remarked, but for the sake of 1 mark, and since I passed quite comfortably, I don't really want to rock the boat.

Answer 1

One thought is that you should've mentioned the regularity conditions for asymptotic normality of the estimates and the $\chi^2$ performance of the likelihood ratio test statistic. These conditions include, informally speaking,

1. the true parameter being in the interior of the parameter space;
2. the log-likelihood really affording Taylor series expansion;
3. i.i.d. data;
4. conditions to interchange some of the derivatives and the integrals/expectations (some sort of uniform boundedness);

and such. See http://www.stat.unc.edu/postscript/rs/ISI89.pdf and http://www.jstor.org/stable/2346086 concerning violations of these conditions. (The simplest example is estimation when the support depends on the parameter value, e.g., $U[0,\theta]$. The MLE $\hat\theta_n=x_{(n)}$ is not asymptotically normal, and an estimator that has a greater asymptotic efficiency in terms of MSE can be constructed.) These are worthy papers to read if you are serious about statistical theory, and many courses on asymptotics do not really wander off far enough into this elephant's graveyard of ML elegance.

Another thought is that may be they really did want you to mention both Wald and score tests. Buse (1982) provides a wonderful review of the relation between the three tests.

Answer 2

Difficult to know what's in the head of a teacher ;-)

The only things that come to my mind are:

1. The source of the results which says that MLE are asymptotically normally distributed (along with checking that hypotheses are satisfied in this case).
2. the dependency of Information $I$ on $n$ which is not clear in your notations (for example writing $\frac{1}{I_n(\theta_0)}$ to emphasize that the estimator is consistent.

Answer 3

You took for granted the result that if $X$ is distributed $N(m,s^2)$ then $(X-m)/s$ is distributed $N(0,1)$. Clearly that is needed for the last step. Also you did not mention the result than the sum of squares of $k$ $N(0,1)$ variables is chi square $n$ degrees of freedom and in your case $n=1$. It is a judgment call whether to accept that you obviously knew that or to consider that formally the proof is incomplete. Do they give partial credit? I think you showed that you knew how to get the hard part (the asymptotic normality of the estimator of $\theta$ with the correct mean and variance). This may appear nit-picky but the question may have been designed for you to demonstrate that you know those facts.