2
$\begingroup$

Let's say I have estimated a parameter $\hat{\theta}$. I want to test the hypothesis that $\hat{\theta}=\theta_0$. My understanding is that I can either use a likelihood ratio test, or just calculate the following z:

$$z=\frac{\hat{\theta}-\theta_0}{\sqrt{\frac{1}{nI(\theta)}}}$$

and test it at the 0.05 significance level. This is a lot easier than using a likelihood ratio test. I also know how to make a confidence interval for $\theta$ using the normal distribution, but I don't know if that's possible with the MLR approach.

My question is: are likelihood ratio tests in situations like this normally used to test hypotheses? For example, if you estimated $\lambda$ for a Poisson, you could calculate a confidence interval using the standard normal distribution and/or do a hypothesis test, but the alternative of doing a likelihood ratio test and using the chi-squared distribution is much more complicated and I'm not sure how to create a confidence interval that way. Is the real use of likelihood ratio tests for more generalized situations where you have nested models and you want to find whether the more complicated model significantly increases the likelihood? Any help is appreciated.

$\endgroup$
3
  • $\begingroup$ Your formula for $Z$ applies only to those extremely specific distributional families where $\theta_0$ is both a parameter of interest and the variance. That pretty much limits it to the Poisson family among standard applications. $\endgroup$
    – whuber
    Dec 20, 2021 at 14:52
  • 1
    $\begingroup$ @whuber I just realized I wrote the wrong formula. I'm updating it to include $I(\theta)$ $\endgroup$
    – fmtcs
    Dec 20, 2021 at 15:13
  • $\begingroup$ @fmtcs Did you mean $I(\widehat{\theta})$ or $I(\theta_0)$? $\endgroup$
    – user277126
    Dec 22, 2021 at 1:11

1 Answer 1

2
$\begingroup$

Suppose we have a one-parameter $(\theta)$ family of distributions. Under certain regularity conditions, if $\theta = \theta_0$, $\sqrt{n}(\widehat{\theta}-\theta_0) \overset{D}{\rightarrow} N(0,I(\theta_0)^{-1})$. Use of this approximation for the hypothesis test/confidence interval, is known as Rao's score test.

By Slutsky's theorem, we also can make use of the approximation $\sqrt{n}(\widehat{\theta}-\theta_0) \overset{D}{\rightarrow} N(0,I(\widehat{\theta})^{-1}).$ Use of this approximation for the hypothesis test/confidence interval, is known as Wald's test.

Let $l_{\theta_0}$ denote the log-likelihood of this distribution evaluated at $\theta_0$, and let $l_{\widehat{\theta}}$ denote the log-likelihood evaluated at the MLE. It may be shown that $-2 \left(l_{\theta_0}-l_{\widehat{\theta}}\right) \overset{D}{\rightarrow} \chi^2_1$. In this case, if we wished to test the hypothesis $H_0: \theta=\theta_0$ vs. $H_1: \theta \ne \theta_0$ at the $\alpha$ significance level, then we need only compute the fitted log-likelihood under both hypotheses and apply a simple function. If this value is larger than the upper $\alpha$ percentile of the $\chi^2_1$ distribution,$\chi^2_{1,\alpha}$, then we reject $H_0$.

Now, the confidence interval for the above likelihood ratio test is defined as $\{\theta_0: -2 \left(l_{\theta_0}-l_{\widehat{\theta}}\right) \le \chi^2_{1,\alpha}\}$ (i.e. those values of $\theta_0$ where we fail to reject the null hypothesis). Clearly, for all $\theta_0$, $l_{\theta_0} \le l_{\widehat{\theta}}$, and one has that $l_{\theta_0} \ge l_{\widehat{\theta}} -\frac{1}{2}\chi^2_{1,\alpha}$. Therefore, we can obtain the confidence interval for $\theta$ based on the likelihood ratio test by plotting the log-likelihood of the distribution and the horizontal line $y=l_{\widehat{\theta}} -\frac{1}{2}\chi^2_{1,\alpha}$. Where these curves intersect along the $x$ axis defines the likelihood ratio confidence interval for $\theta$.

Here is a quick example using the Poisson family:

set.seed(555)
n=20
lambda=3
alpha=.05
x = rpois(n,lambda)
mle.x = mean(x)

ll.pois = function(x,lambda){
  return(sum( log( dpois( x, lambda) ) ) )
}
ll.pois.vec = Vectorize(ll.pois,"lambda")
lambda=seq(1.75,5,length=1000)
y = ll.pois.vec(x,lambda)
h = ll.pois(x,mle.x)-.5*qchisq(1-alpha,1)
plot(y~lambda,type="l",,xlab=expression(lambda),main="Log-likelihood of Poisson Dist.", ylab = "Log-Likelihood")
abline(h=h,col=2,lty=2)
lambda[min(which(y>=h))] 
lambda[max(which(y>=h))] 
abline(v=lambda[min(which(y>=h))] ,lty=2)
abline(v=lambda[max(which(y>=h))] ,lty=2)

enter image description here

$\endgroup$
1
  • $\begingroup$ The visualization really brings this point home! Thank you for this great answer! $\endgroup$
    – stats_noob
    Aug 13, 2022 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.