Rao's Score/Lagrange Multiplier Test most powerful when $\theta$ close to $\theta_0$?

Question

I was reading over the documentation for the Rao's Score/Lagrange Multiplier Test on wikipedia and ran across this:

Rao's score test is a statistical test of a simple null hypothesis that a parameter of interest $\theta$ is equal to some particular value $\theta_0$. It is the most powerful test when the true value of $\theta$ is close to $\theta_0$. The main advantage of the Score-test is that it does not require an estimate of the information under the alternative hypothesis or unconstrained maximum likelihood. This constitutes a potential advantage in comparison to other tests, such as the Wald test and the generalized likelihood ratio test (GLRT). This makes testing feasible when the unconstrained maximum likelihood estimate is a boundary point in the parameter space.

In statistics class we were told the Likelihood Ratio test is the best test to evaluate nested logistic regression models as it utilizes more of your data than Wald or the Score test. They mentioned that the other tests could be useful in certain situations, which I guess is being described here.

I've looked through gung's very helpful discussion of the trinity of tests. As far as I can make out the Score method is most useful when the likelihood that the other two tests rely on approaches large numbers/infinity, but am still confused as to what is being said in the above quote. The italicized portion of the quote is what is most confusing.

Could someone break down the language a bit more, talk about thetas, the likelihood and parameter space, and outline in which type of applied situations Rao's score is more powerful? Thanks!

Answer 1

There is a lot of misunderstanding in your interpretation of what you've read.

Likelihood Ratio test is the best test to evaluate nested logistic regression models as it utilizes more of your data than Wald or the Score test.

The Likelihood Ratio Test does not "use more data" than the other test. Nested logistic regression models is but a special case of situations where the LRT applies. Any simple or composite hypothesis is suitable for testing using a LRT. The Neyman Pearson lemma is correct: LRTs are Uniformly Most Powerful.

As far as I can make out the Score method is most useful when the likelihood that the other two tests rely on approaches large numbers/infinity,

I don't understand what you're trying to say here. The Score test (not the score method, which could be confused with Fisher scoring, a method of estimating GLMs using their score function) can be evaluated when other tests fail. Other tests fail when the unconstrained maximum likelihood leads to parameter estimates which lie on the boundary of the parameter space, therefore the information singular. For logistic regression models, you may estimate an infinite odds ratio, and the LRT and Wald tests yield infinite valued test statistics whereas the Score test is finitely valued.

The score test works by inspecting the slope of the likelihood function at the null hypothesis (no need to evaluate the information at the alternative). If the likelihood function has a large slope, there is evidence that the null hypothesized value is far from the maximum likelihood estimate. Did you see the picture? If the likelihood function slopes upward indefinitely for a limit to the right or left, the Wald and LR stat cannot be evaluated (or more correctly, are infinite), but the score statistic is available.