4
$\begingroup$

I'm trying to wrap my head around why it is intuitive that (under certain conditions) the likelihood ratio statistic follows a chi-squared distribution, asymptotically.

I've looked at the excellent answers to this and this question and can follow the derivations that use a Taylor expansion around the Maximum Likelihood Estimate and a Central Limit Theorem. However, once I step back, I don't find this result very intuitive.

I'm hoping there exists a heuristic, high-level argument for why the log of the ratio of the two likelihoods in question (or alternatively the difference of their logs) would asymptotically behave like (the sum of) one (or more independent) squared standard normal random variable(s).

$\endgroup$
3
  • 3
    $\begingroup$ The LR is not "the ratio of ... two log likelihoods." Instead, it is the logarithm of the ratio. There's a huge difference! $\endgroup$
    – whuber
    Nov 12 '21 at 18:36
  • $\begingroup$ @whuber great spot, sorry for the typo! $\endgroup$
    – Matsaulait
    Nov 12 '21 at 18:40
  • $\begingroup$ Okay, I'm glad that you have the definition correct--that makes this a useful question. $\endgroup$
    – whuber
    Nov 12 '21 at 19:09
4
$\begingroup$

My intuition is this: i) For a normal model the likelihood itself has the appearance of a normal distribution, and the likelihood ratio test statistic is the square of a t-statistic, i.e. an F-statistic. Asymptotically, this F-statistic follows a chi-square distribution in repeated sampling. ii) When the mild regularity conditions are met for a non-normal model the profile likelihood itself will, with increasing sample size, have the appearance of a normal distribution suggesting that, with increasing sample size, the likelihood ratio test statistic will follow a chi-square distribution in repeated sampling. Even when the sample size is modest and the profile likelihood does not have the precise appearance of a normal distribution, the chi-square approximation to the likelihood ratio test statistic often still works incredibly well. Here is a simple example using exponentially distributed data that compares p-values and confidence intervals of all levels using the chi-square approximation to p-values and confidence intervals of all levels using the exact sampling distribution of the likelihood ratio test statistic.

I prefer to view the likelihood ratio test statistic as a monotonic transformation of the maximum likelihood estimator. Treating the profiled nuisance parameters as known, we could instead derive or approximate the sampling distribution of the MLE and use this to construct p-values and confidence intervals (think of inverting a binomial CDF for inference on a proportion). The Wald, score, and likelihood ratio tests are approximations to referencing the CDF of the MLE.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.