1
$\begingroup$

Is likelihood ratio test ($F$-test) of significance of difference of two linear models the same as chi-square test of difference of $-2\log L$?

SAS PROC GLM produces $F$-statistics and PROC MIXED $-2\log L$.

$\endgroup$

1 Answer 1

5
$\begingroup$

I think that you might be confusing an extra-sum-of-squares F-test with a likelihood ratio test. Although, both are used to compare two models.

A likelihood ratio statistic, denoted by $\Lambda$, is given by

$$\Lambda = \frac{L\text{(reduced model})}{L(\text{full model})}$$

Taking $-2\log\Lambda$ produces a statistic that has $\chi^2_{d.f(\text{reduced model})-d.f(\text{full model})}$ distribution. That is to say that taking $-2\log$ of the $\Lambda$ gives you a $\chi^2$ distribution.

I have not used SAS so I cannot comment on the output, but I hope that I have been able to answer your question.

Note: that $\Lambda$ is equivalent to your L


Janne: For linear regression you could use either the likelihood ratio test or the extra-sum-squares F-test and you should end up with the same p-value. Despite, this they are not the same thing.

As has been mentioned above the likelihood ratio test produces a statistic that has $\chi^2_{d.f(\text{reduced model})-d.f(\text{full model})}$ distribution. Where as an extra-sum-of-squares F-test, given by

$$F = \frac{(SSR_{\text{reduced model}}-SSR_{\text{full model}})/d.f_{\text{reduced model}} - d.f_{\text{full model}}}{\hat{\sigma}^2_\text{full model}}$$

producing a statistic that has $F_{d.f(\text{reduced model})-d.f(\text{full model}),d.f(\text{full model})}$ distribution. Where SSR is the sum of squared residuals and $\hat{\sigma}^2$ is our standard estimate.

$\endgroup$
4
  • $\begingroup$ Thank You. So Chi2-test for comparing two models is not the same as F-test for the same purpose. The p-value I mean... $\endgroup$
    – Janne
    Nov 24, 2011 at 18:56
  • 1
    $\begingroup$ I think the difference in d.o.f. for the $\chi^2$ should read the other way around (full - reduced). $\endgroup$
    – chl
    Nov 24, 2011 at 22:25
  • $\begingroup$ @chl it is only full - reduced if you take the -2 into the equation. $\endgroup$
    – user7045
    Nov 25, 2011 at 8:22
  • 1
    $\begingroup$ The reduced model has less parameters, thus lower df. It turns out that $df_\text{reduced}-df_\text{full}<0$. $\endgroup$
    – chl
    Nov 25, 2011 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.