7
$\begingroup$

When we say the likelihood ratio statistic and the Wald statistic of a set of binomial distributions are asymptotically equivalent, do we mean that the sampling distributions of the two statistics will get close as we take larger samples of both statistics from a fixed set of binomial distribution?

Or does this mean that the two sampling distributions will converge as we increase the number of trials $n$ of the binomial distribution of interest?

I am slightly confused as to what $n$ (as in $n\rightarrow \infty$) stands for in this case.

$\endgroup$

1 Answer 1

6
$\begingroup$

Either the number of binomial trials or the number of observations will do; usually we think of this result as applying more generally than binomial data and so think of the number of observations as the $n\to\infty$.

It's also important to note that the asymptotic equivalence is local. Suppose 0 is the null value of $\theta$. If you set $\theta=\theta_A=\neq 0$ and take $n$ observations with that value of $\theta$, $n\to\infty$, there is no guarantee that the the test statistics will approach each other. The standard result is that if you take have a sequence values $\theta_n=h/\sqrt{n}$ and take $n$ observations with $\theta=\theta_n$, then as $n\to\infty$ the score, Wald, and likelihood ratio tests will converge in probability to the same random variable.

Here's the picture: on a graph with the score (derivative of loglikelihood) on the $y$-axis and $\theta$ on the $x$-axis, the Wald chi-squared statistic is twice the area of the blue triangle. The score chi-squared statistic is twice the area of the red triangle, and the likelihood ratio chi-squared statistic is twice the grey area under the curve.

triangles of Wald,LRT, score

With $n\to\infty$ and $\theta_n=h/\sqrt{n}$, we're zooming in to this picture. The curve locally asymptotically approaches a straight line and the three coloured areas become the same enter image description here

But if you fix $\theta\neq 0$ and just increase $n$, the picture doesn't change. It still just looks like enter image description here and there's no asymptotic equivalence.

$\endgroup$
9
  • $\begingroup$ sorry, just to clarify more, so if I artificially generate a large number of likelihood ratio and wald test statistics from a given set of binomial distribution while keeping the number of samples drawn from the binomial distributions to be small, will the distributions of the simulated likelihood ratio and wald test statistics in this case be very close to one another? (i.e. due to the asymptotic equivalence?) thank you, $\endgroup$
    – HDB
    Commented Oct 28, 2020 at 3:33
  • 2
    $\begingroup$ Thomas's answer was great but also note that the result that is being discussed is not specific to the binomial. It holds whenever the Wald test and the LR test are applicable. $\endgroup$
    – mlofton
    Commented Oct 28, 2020 at 4:20
  • 1
    $\begingroup$ This kind of concept should be explained with this kind of graphic more often. Does $h$ represent either of the three statistics, that is Wald, score and likelihood ratio? $\endgroup$ Commented Oct 28, 2020 at 9:20
  • 1
    $\begingroup$ +1: could you provide more detail on when the picture "applies"? Does it also work for, say, nonlinear models? $\endgroup$ Commented Nov 5, 2020 at 14:19
  • 1
    $\begingroup$ Yes, it does. Not for mixed models so much. $\endgroup$ Commented Nov 5, 2020 at 21:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.