Likelihood ratio test multiplying by 2

Question

Regarding the likelihood ratio test. I can't seem to find an answer to why we multiply the log likelihood ratio's with 2. On wikipedia it says we multiply by 2, in order to mathematically, say they are approximately chi squared distributed (Wilks Theorem). However, I can't seem to understand why multiplying by 2 is necessary in order to make the LRT approximately chi squared distributed?

Answer 1

It is relatively simple to see where the 2 comes from in the one parameter case. Let $\ell(\theta_0)$ be the log-likelihood for the true value of the parameter $\theta$. A Taylor Expansion around the MLE gives:

$\ell(\theta_0)\approx\ell(\hat\theta)+\ell'(\hat\theta)(\theta_0-\hat\theta)+\frac{1}{2}\ell''(\hat\theta)(\theta_0-\hat\theta)^2$

Now re-arrange terms. Since, $\ell'(\hat\theta)=0$ by definition of the MLE:

$\ell(\theta_0)-\ell(\hat\theta)\approx\frac{1}{2}\ell''(\hat\theta)(\hat\theta-\theta_0)^2$

Multiplying by -2 on both sides we have

$-2(\ell(\theta_0)-\ell(\hat\theta))\approx -\ell''(\hat\theta)(\hat\theta-\theta_0)^2$

The term on the left is the usual LR test statistic. The term on the right hand side is asymptotically $\chi^2_1$ under regularity conditions on the underlying distribution and by the Central Limit/Slutsky's Theorem. In this case, the df=1 since the null space has dimension 0 and the alternative space has dimension 1. Wilks' Theorem extends this to the multiparameter case, but the underlying mathematics are more complex.

Answer 2

Suppose we did not have $-2$ for the likelihood ratio. How could you make a test for the ratio? Do you know the distribution of the ratio?

To answer this question we can look at Wilk's theorem:

As the sample size approaches ${\displaystyle \infty }$, the distribution of the test statistic $-2 \log(\Lambda)$ asymptotically approaches the chi-squared $\chi ^{2}$ distribution under the null hypothesis ${\displaystyle H_{0}}$. Here, ${\displaystyle \Lambda }$ denotes the likelihood ratio, and the ${\displaystyle \chi ^{2}}$ distribution has degrees of freedom equal to the difference in dimensionality of ${\displaystyle \Theta }$ and ${\displaystyle \Theta _{0}}$, where ${\displaystyle \Theta }$ is the full parameter space and ${\displaystyle \Theta _{0}}$ is the subset of the parameter space associated with ${\displaystyle H_{0}}$. This result means that for large samples and a great variety of hypotheses, a practitioner can compute the likelihood ratio ${\displaystyle \Lambda }$ for the data and compare ${\displaystyle -2\log(\Lambda )}$ to the ${\displaystyle \chi ^{2}}$ value corresponding to a desired statistical significance as an approximate statistical test.

Therefore we need to change the likelihood-ratio by adding the $\log$ function and multiply it to $-2$ for using Wilk's theorem for making a test on its final result. As we know the distribution of the changing formula based on its asymptotic behavior.