When to use maximum likelihood ratio

Question

Let's say I have estimated a parameter $\hat{\theta}$. I want to test the hypothesis that $\hat{\theta}=\theta_0$. My understanding is that I can either use a likelihood ratio test, or just calculate the following z:

$$z=\frac{\hat{\theta}-\theta_0}{\sqrt{\frac{1}{nI(\theta)}}}$$

and test it at the 0.05 significance level. This is a lot easier than using a likelihood ratio test. I also know how to make a confidence interval for $\theta$ using the normal distribution, but I don't know if that's possible with the MLR approach.

My question is: are likelihood ratio tests in situations like this normally used to test hypotheses? For example, if you estimated $\lambda$ for a Poisson, you could calculate a confidence interval using the standard normal distribution and/or do a hypothesis test, but the alternative of doing a likelihood ratio test and using the chi-squared distribution is much more complicated and I'm not sure how to create a confidence interval that way. Is the real use of likelihood ratio tests for more generalized situations where you have nested models and you want to find whether the more complicated model significantly increases the likelihood? Any help is appreciated.

Answer 1

Suppose we have a one-parameter $(\theta)$ family of distributions. Under certain regularity conditions, if $\theta = \theta_0$, $\sqrt{n}(\widehat{\theta}-\theta_0) \overset{D}{\rightarrow} N(0,I(\theta_0)^{-1})$. Use of this approximation for the hypothesis test/confidence interval, is known as Rao's score test.

By Slutsky's theorem, we also can make use of the approximation $\sqrt{n}(\widehat{\theta}-\theta_0) \overset{D}{\rightarrow} N(0,I(\widehat{\theta})^{-1}).$ Use of this approximation for the hypothesis test/confidence interval, is known as Wald's test.

Let $l_{\theta_0}$ denote the log-likelihood of this distribution evaluated at $\theta_0$, and let $l_{\widehat{\theta}}$ denote the log-likelihood evaluated at the MLE. It may be shown that $-2 \left(l_{\theta_0}-l_{\widehat{\theta}}\right) \overset{D}{\rightarrow} \chi^2_1$. In this case, if we wished to test the hypothesis $H_0: \theta=\theta_0$ vs. $H_1: \theta \ne \theta_0$ at the $\alpha$ significance level, then we need only compute the fitted log-likelihood under both hypotheses and apply a simple function. If this value is larger than the upper $\alpha$ percentile of the $\chi^2_1$ distribution,$\chi^2_{1,\alpha}$, then we reject $H_0$.

Now, the confidence interval for the above likelihood ratio test is defined as $\{\theta_0: -2 \left(l_{\theta_0}-l_{\widehat{\theta}}\right) \le \chi^2_{1,\alpha}\}$ (i.e. those values of $\theta_0$ where we fail to reject the null hypothesis). Clearly, for all $\theta_0$, $l_{\theta_0} \le l_{\widehat{\theta}}$, and one has that $l_{\theta_0} \ge l_{\widehat{\theta}} -\frac{1}{2}\chi^2_{1,\alpha}$. Therefore, we can obtain the confidence interval for $\theta$ based on the likelihood ratio test by plotting the log-likelihood of the distribution and the horizontal line $y=l_{\widehat{\theta}} -\frac{1}{2}\chi^2_{1,\alpha}$. Where these curves intersect along the $x$ axis defines the likelihood ratio confidence interval for $\theta$.

Here is a quick example using the Poisson family:

set.seed(555)
n=20
lambda=3
alpha=.05
x = rpois(n,lambda)
mle.x = mean(x)

ll.pois = function(x,lambda){
  return(sum( log( dpois( x, lambda) ) ) )
}
ll.pois.vec = Vectorize(ll.pois,"lambda")
lambda=seq(1.75,5,length=1000)
y = ll.pois.vec(x,lambda)
h = ll.pois(x,mle.x)-.5*qchisq(1-alpha,1)
plot(y~lambda,type="l",,xlab=expression(lambda),main="Log-likelihood of Poisson Dist.", ylab = "Log-Likelihood")
abline(h=h,col=2,lty=2)
lambda[min(which(y>=h))] 
lambda[max(which(y>=h))] 
abline(v=lambda[min(which(y>=h))] ,lty=2)
abline(v=lambda[max(which(y>=h))] ,lty=2)