Why is it the likelihood ratio test under Neyman-Pearson and the Generalized Likelihood Ratio Test lead to the same results?

Question

Given data $X_1, \ldots, X_N$, suppose I wanted to test $H_0: \theta \in \Theta_0$ vs. $H_1: \theta \in \Theta_1$, letting $\Theta_0 \cup\theta_1 = \Theta$.

Under Neyman Pearson, we have:

$$ \Lambda_a = \dfrac{max_{\theta \in \Theta_0}L(x_1,\ldots,x_n|\theta)}{max_{\theta \in \Theta_1}L(x_1,\ldots,x_n|\theta)} $$

Under the Generalized Likelihood Ratio Test, we have:

$$ \Lambda_b = \dfrac{max_{\theta \in \Theta_0}L(x_1,\ldots,x_n|\theta)}{max_{\theta \in \Theta}L(x_1,\ldots,x_n|\theta)} $$

I have been told these lead to the same criterion for null rejection, why is that the case?

Answer 1

The Neyman-Pearson Lemma says that when you have just a single-point null and single-point alternative that the most powerful test is the test based on the likelihood ratio (at least in theory; it might only be a randomised test). You have $\max$ in your notation. There is no $\max$ for the Neyman-Pearson Lemma. There is nothing to $\max$ over. Having something to $\max$ over (composite hypotheses) is exactly what causes problems.

Typically, there is no most powerful test when comparing two composite hypotheses; there are many possible tests and different ones will have higher power against different alternatives. You ask Theory for suggestions, and Theory shrugs unhelpfully. Because we still need to choose tests, it is tempting to keep using likelihood ratio tests.

Suppose $\theta_0$ is a point in the null and $\theta_1$ is a point in the alternative, and $L_1(\theta_1)/L_0(\theta_0)$ is the most powerful test comparing them. If you happen to get $\hat\theta=\theta_0$ under the null and $\hat\theta=\theta_1$ under the alternative, it seems reasonable that $L_1(\theta_1)/L_0(\theta_0)$ will still be a good test statistic, even if perhaps not the best possible one. That's the Generalized Likelihood Ratio Test. It can be written more formally as $$\frac{\max_{\theta\in\Theta_1} L_1(\theta)}{\max_{\theta\in\Theta_0} L_0(\theta)}$$ This is not the Neyman-Pearson test (although it looks a bit like it). The test statistic is what the Neyman-Pearson test statistic would be if you pretended you'd chosen $\theta_0$ and $\theta_1$ in advance, rather than from the data. The rejection region will typically be different, because you need to acknowledge the data-dependent choices of $\theta_0$ and $\theta_1$ [[1]]

Very occasionally, there is a uniformly most powerful test for composite hypotheses. One of the main settings where there is one and we can easily work out what it is, is the setting of the Karlin-Rubin theorem. In that setting we have a one-sided test for a one-dimensional parameter and the Neyman-Pearson test of a specific $\theta_0$ happens to be the same regardless of $\theta_1$. And in that setting, the uniformly most powerful test is the same as the generalised likelihood ratio test. There's no data dependence in the test statistic or in the rejection region because, well, there isn't; go look at the proof of the Karlin-Rubin theorem.

Usually, there is no uniformly most powerful test, and so the question of whether it agrees with the generalised likelihood ratio test doesn't arise. I don't actually know if there are examples where there's a UMP test that isn't the generalised likelihood ratio test but I'd be a bit surprised if there are any that don't look massively contrived.

[1] though in fact people will often assume the impact of these choices goes away in large enough samples and that they have large enough samples, because that makes life easier. In particular, you get the usual log likelihood ratio tests with asymptotic chi-squared distributions that way.

Answer 2

They're only the same under the assumptions of the Karlin-Rubin theorem: that is that the likelihood is monotonic in $\theta$. By extension, the criterion must be selected by choosing the critical value that leads to the uniformly most powerful test. If you set an $\alpha$-level and establish the critical value based on the distribution of the likelihood under the null hypothesis, it will be the same value in $\Theta_1$ regardless.

However violate Karlin-Rubin's assumptions and you will NOT NECESSARILY identify the same critical regions.

Answer 3

As @ThomasLumley points out, neither of these is the Neyman–Pearson test statistic: they're different definitions of a generalized likelihood ratio test statistic; the former is perhaps the more direct generalization of the likelihood ratio for simple hypotheses, $\frac{L(\theta_0)}{L(\theta_1)}$. Note that when $\Theta_0$ is of lower dimensionality than $\Theta_1$ the two statistics will be equal; in any case they'll only differ when $\hat\theta \in \Theta_0$, which isn't where you'll usually be constructing rejection regions.

The case where $\Theta_0$ is of lower dimensionality than $\Theta_1$ arises commonly: $\theta=(\phi, \psi)$, where $\phi$ is the parameters of interest, & $\psi$ is nuisance parameters; with the null hypothesis formulated as $H_0:\theta_0 \in \Theta_0\equiv\phi=\phi_0, \psi\in\Psi$. In the numerator of the generalized LRT you then maximize over $\psi\in\Psi$, & in the denominator it makes no odds whether you say you're maximizing over $\phi \in\Phi, \psi\in\Psi$ or $\phi \in\Phi-\phi_0, \psi\in\Psi$. The region of the parameter space carved out by the null has no interior that's any distance from its complement.