# Justification for use of $\chi^2(1)$ in Wald and score test

In a recent exam, we were asked to justify the use of the $\chi^2(1)$ distribution in performing the Wald or Rao's score test. There was only 1 mark for this (approx. 2.5 mins worth of time). My answer was

The Wald and score test statistics are based on various approximations to the log-likelihood ratio, which are valid and equivalent in large samples when $H_0$ is true. For example, the approximation for the 2nd Wald statistic is

$$2\log(LR)\simeq (\hat{\theta}_n-\theta_0)^2E\{-\ell''(\theta)\}|_{\theta_0} =(\hat{\theta}_n-\theta_0)^2I(\theta_0)$$

where $I(\theta_0)$ is the Fisher information at $\theta_0$. Then, using asymptotic normality,

$$\hat{\theta}_n\approx N \left ( \theta_0,\frac{1}{ni(\theta_0)}\right )=N \left ( \theta_0,\frac{1}{I(\theta_0)}\right )$$

which yields $(\hat{\theta}_n-\theta_0)\sqrt{I(\theta_0)} \approx N(0,1)$

Since $2\log(LR)$ is approximately equal to the square of the LHS of this when $n$ is large, it is approximately distributed as the square of a standard normal random variable, that is, as $\chi^2(1)$

The marker wrote "insufficient" and I got zero for this. As it is a summative exam, they won't give any feedback, nor engage in any discussion about it. I was wondering if anyone here can explain what I have missed or where I went wrong. I'm not very good with latex so I hope I didn't make any mistakes in the typing !

This is for an elective module in statistical theory in the final year of an undergraduate maths degree. Thanks !

Edit: There is a formal procedure to have my script remarked, but for the sake of 1 mark, and since I passed quite comfortably, I don't really want to rock the boat.

• (1) Perhaps they were expecting you to mention the approximation using $I(\hat\theta_n)$; (2) Are you sure it is Rao's or Wald and not Rao's and Wald? (3) Marking is a rigid process, perhaps you failed to justify some steps like using a second order approximation.
– user10525
Commented May 26, 2012 at 14:57
• @Procrastinator thanks for your comment. In the first part of the question we had to find the MLE from a given pdf [7 marks]. The second part was to give expressions for the Wald statistics and the score statistic [6 marks]. Then we had to describe the procedure to obtain the relevant p-value [2 marks]. Finally we had to justify the distribution used [1 mark]. I got all but the final mark. Commented May 26, 2012 at 15:22
• @P Sellaz Did you also proved this result for the score statistic?
– user10525
Commented May 26, 2012 at 15:26
• @Procrastinator thanks again. No, I just picked the 2nd Wald statistic as an example...... Commented May 26, 2012 at 17:33

One thought is that you should've mentioned the regularity conditions for asymptotic normality of the estimates and the $\chi^2$ performance of the likelihood ratio test statistic. These conditions include, informally speaking,

1. the true parameter being in the interior of the parameter space;
2. the log-likelihood really affording Taylor series expansion;
3. i.i.d. data;
4. conditions to interchange some of the derivatives and the integrals/expectations (some sort of uniform boundedness);

and such. See http://www.stat.unc.edu/postscript/rs/ISI89.pdf and http://www.jstor.org/stable/2346086 concerning violations of these conditions. (The simplest example is estimation when the support depends on the parameter value, e.g., $U[0,\theta]$. The MLE $\hat\theta_n=x_{(n)}$ is not asymptotically normal, and an estimator that has a greater asymptotic efficiency in terms of MSE can be constructed.) These are worthy papers to read if you are serious about statistical theory, and many courses on asymptotics do not really wander off far enough into this elephant's graveyard of ML elegance.

Another thought is that may be they really did want you to mention both Wald and score tests. Buse (1982) provides a wonderful review of the relation between the three tests.

• Thank you. Could you expand a little more about what you mean by "$\chi^2$ performance of the likelihood ratio test statistic" ? In this module, the regularity conditions are discussed but not in great detail. I believe the information given in the question (the pdf and the statement about the data collected) imply that the conditions are satisfied. Perhaps I should have just said (bearing in mind that this is all for just 1 mark) "from the pdf definition and data the regularity conditions for asymptotic normality are satisfied" ? Commented May 28, 2012 at 8:57
• When the regularity conditions are violated, the asy distribution of the LR test statistic is no longer $\chi^2$, as some of the references I gave discuss. It may become a sum of differently weighted $\chi^2$, a sum of $\chi^2$ with different degrees of freedom, or a supremum of a $\chi^2$ process. Commented Jun 9, 2012 at 15:19

Difficult to know what's in the head of a teacher ;-)

The only things that come to my mind are:

1. The source of the results which says that MLE are asymptotically normally distributed (along with checking that hypotheses are satisfied in this case).
2. the dependency of Information $I$ on $n$ which is not clear in your notations (for example writing $\frac{1}{I_n(\theta_0)}$ to emphasize that the estimator is consistent.
• Thanks. I suspect either of these points could be it. We've done the proofs of these. It just seems like a lot of work for 1 mark. TBH, had I covered these, I would have written more for this 1 mark than for the 13 marks of the first 2 parts combined (see my comment in reply to procrastinator's first comment on my question). Having said that, have come unstuck in making assumptions of what an examiner is expecting based on the marks available for a question - but in an exam, I always allot my time based on the marks available - I don't know any other way to proceed. Commented May 26, 2012 at 17:42

You took for granted the result that if $X$ is distributed $N(m,s^2)$ then $(X-m)/s$ is distributed $N(0,1)$. Clearly that is needed for the last step. Also you did not mention the result than the sum of squares of $k$ $N(0,1)$ variables is chi square $n$ degrees of freedom and in your case $n=1$. It is a judgment call whether to accept that you obviously knew that or to consider that formally the proof is incomplete. Do they give partial credit? I think you showed that you knew how to get the hard part (the asymptotic normality of the estimator of $\theta$ with the correct mean and variance). This may appear nit-picky but the question may have been designed for you to demonstrate that you know those facts.

• Thanks. Among my peers whom I spoken to, no one got any credit for this part of the question. I don't know if they give half marks or not. I feel that your points are of the variety that we could call "obvious" on our course - while for the points made by @gui11aume I can't believe we were expected to do all that for 1 mark, so I am left thinking that your points may well be it. Commented May 26, 2012 at 17:50
• It could be a combination of our explanations. But you know your instructor. So you are probably in the best position to figure what his rationale is likely to be. I think the important thing and the reason you brought the question here was to be assured that there was no major mistake in your argument. These minor things are not important but may serve as a lesson to dot is and cross ts on test solutions. Commented May 26, 2012 at 18:02
• The thing is, to dot all the is and cross all the ts would require too much time on a 1 hour exam when this part was worth only 1 mark. The cynical among us believe it could be the instructors 'impossible question', as no one in their right mind would go into such detail for one mark. He has mentioned on more than one occasion that no one ever got 100% on this module. When looking through past papers there is nothing in any of them that stands out as exceptionally difficult for the marks available, so maybe he just sets a marking scheme to ensure that certain questions can't get full credit. Commented May 26, 2012 at 18:09
• I get your point and I am very sympathetic. In this case though I don't see it taking much time to add two short sentences to cover the gaps in the proof. Commented May 26, 2012 at 18:24
• For your 2 points, yes, it wouldn't be too onerous, but what about the 2 points made by @gui11aume ? I think I'll write out a model answer with all the additional points that have been raised in this thread and see how it looks. It will be a good consolidation exercise anyway...The main point of it all, is to learn, and a mark here or there isn't really relevant; the good thing about this kind of post-mortem is that it makes me think even more about it, which is great :) Thanks again ! Commented May 26, 2012 at 18:36