2
$\begingroup$

I am trying to understand a fundamental property of the Likelihood Ratio Test (LRT).

For simplicity, the following example is framed as binomial data, which of course could be solved using a simple hypothesis test (the binomial test). But for the sake of the argument, please consider it from the LRT perspective, since it is a nice simplification of more complicated scenarios.

Let $X$ ~ binomial$(n,p)$. Let $p_0 = 0.2$.

I would like to test that $H_0: p \in (0 , p_0)$ vs. $H_1: p \in (p_0 , 1)$.

Then my LRT test statistic is, as usual:

$\Lambda = - 2 \cdot \log \frac{sup \{L(\theta | x) : \theta \in (0 , p_0) \}}{sup \{L(\theta | x) : \theta \in (0,1)\}} $

I have read that the likelihood ratio test is not universal. I've read that it fails when:

  1. the parameter is constrained to be on the boundary (e.g. $N_2( [\mu_1 , \mu_2 ] , \Sigma)$ where $\mu_1, \mu_2 \ge 0$ and $H_0: \mu_1 = \mu_2 = 0$ ).
  2. The parameters must be on the "interior" of the parameter space.

In these cases, the LRT statistic $\Lambda$ is not asympotitcally $\chi^2$ distributed.

In the topological sense, my $H_0$ constrained parameter space $(0,p_0)$ is indeed an interior set to $(0,1)$. Further, I am not constraining $p$ to be on the boundary of $(0,1)$. Seems like I should be OK, right?

Is the LRT statistic above still valid ? i.e. is $\Lambda$ above still asymptotically ~ $\chi^2$ ?

Based on my reasoning about the interior point/constraints above, I believe the answer is yes.

Lastly: what is the degrees of Freedom of my LRT test?

I believe it is 1, intuitively.

$\endgroup$
  • $\begingroup$ >$H_1: p \in (p_0 , 1)$ you meant $H_1: p \in (0 , 1)$ right ? $\endgroup$ – brumar May 21 '15 at 20:24
  • $\begingroup$ Sure. This is a moot point. The form of the LRT statistic $\Lambda$ as I wrote in the question does not change. Conventionally, you state $H_0$ and $H_1$ as disjoint. So you state $H_1$ as $(p_0,1)$. But for the LRT, you want $H_0$ nested in $H_1$, so you "use" $(0,1)$. $\endgroup$ – cmo May 21 '15 at 20:55
3
$\begingroup$

The chi-squared limiting distribution is valid only for a special type of composite hypotheses. $H_0: \theta_1=\theta_1^0,\ldots,\theta_r=\theta_r^0,\theta_{r+1},\ldots,\theta_k$, that is when the first $r$ parameters are specified and the rest are not, versus $H_a: \theta_1,\ldots,\theta_k$ that leaves all of them unspecified.

Your null hypothesis does not have that form, and it is easy to see that the limiting distribution is not chi-squared under the null hypothesis. Suppose $p$ is small (less than 0.2), and $n$ is large. Then the maximum likelihood estimate will be almost always less than 0.2 either under the null or the without restriction. Then your test statistic will be almost always 0! In fact, its limiting distribution is degenerate.

In general, likelihood-ratio tests are not convenient for one-sided alternatives, because the direction of the difference is obliterated by the squaring. However the correct way for developing a one-sided likelihood-ratio test is to note that $p=0.2$ is closest to the alternative, so we test $H_0: p=0.2$ versus $H_a: p\geq 0.2$. In this case, however, the value of interest is on the edge of the parameter space, so the limiting distribution is not chi-squared. In fact, for this case it is a mixture distribution of $0.5 I({0}) + 0.5\chi^2_1$.

$\endgroup$
  • $\begingroup$ I understand. What hypothesis test is sufficient to test $H_0: p \le p_0$ vs $H_1: p > p_0 $ ? $\endgroup$ – cmo May 23 '15 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.