9
$\begingroup$

When we are performing maximum likelihood estimation (MLE) to estimate parameters, the fit function is often to -2 * LL, rather than just LL. I also see this "-2LL" term expressed as "deviance". Why do we multiply log likelihood times -2? Is it if we are trying to minimize the function rather than maximize? What is the purpose of it being multiplied by (a) a negative number, and (b) a "2"?

Some locations where "-2 log likelihood" is expressed are below:

$\endgroup$
5
  • $\begingroup$ Can you please provide an example of where you see the 2 arise? $\endgroup$ – Arya McCarthy Apr 10 at 16:52
  • $\begingroup$ Sure-- I just added some links below where -2 LL is expressed $\endgroup$ – JElder Apr 10 at 16:57
  • $\begingroup$ The sources you provide does not estimate parameters with -2*LL it is likelihood ratio test . Deviance is a measure for likelihood ratio test, 2 is coming from chi square distrb.(its multiplied by rather than square because ratio is ratio of log likelihoods) You estimate parameters by minimizing -LL, multiplying it with any constant number will not change the results. Here the link for different explanations LRT: sciencedirect.com/topics/mathematics/… $\endgroup$ – CheeseBurger Apr 10 at 17:12
  • 4
    $\begingroup$ This factor is linked with Wilks' theorem, as the factor 2 brings the likelihood ratio to be asymptotically $\chi^2_\Delta$. The maximum obviously does not change. $\endgroup$ – Xi'an Apr 10 at 17:30
  • 1
    $\begingroup$ Hint: Write the loglikelihood for a linear regression model, and multiply by $-2$. What do you get? $\endgroup$ – kjetil b halvorsen Apr 11 at 0:41
11
$\begingroup$

Why do we multiply log likelihood times -2 when conducting MLE? We really don't. The -2 was not about parameter estimation; for that, we'd just use the (negative log-)likelihood. It was about hypothesis testing.


Your intuition about negation is correct. Traditionally, in the optimization literature, we minimize functions. It's easy enough to convert a maximization problem into a minimization problem by negating the objective. The parameters that maximize the log-likelihood are the ones that minimize the negative log-likelihood.


You've shown some links that use the quantity -2LL in the specific case of linear regression. There's a computational reason for this and a statistical reason.

  • The computational reason (which is weaker; more of an 'it doesn't matter'). An objective multiplied by a scalar constant will have the same optimum. In the Gaussian log-likelihood, every term is a fraction with denominator 2. So why bother dividing? By including the -2, you don't have to divide every term by 2. (Not that computers have much trouble with dividing by powers of 2...)

  • The statistical reason (which argues for a meaningful benefit of the -2). This quantity, as you note, is called the deviance. The -2 factor is useful for statistical hypothesis testing. In a likelihood ratio test, this helps you to compute a $p$-value. Quoting the Wikipedia article on likelihood ratio tests:

    Multiplying by −2 ensures mathematically that (by Wilks' theorem) ${\displaystyle \lambda _{\text{LR}}}$ converges asymptotically to being χ²-distributed if the null hypothesis happens to be true.

    To add context, I'll also quote two of the articles you linked: first

    LR chi2(3) – This is the likelihood ratio (LR) chi-square test. The likelihood chi-square test statistic...This is minus two (i.e., -2) times the difference between the starting and ending log likelihood.

    and second:

    Multiplying it by -2 is a technical step necessary to convert the log-likelihood into a chi-square distribution, which is useful because it can then be used to ascertain statistical significance. Don't worry if you do not fully understand the technicalities of this.

    They both give the same message as the Wikipedia article.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.