10
$\begingroup$

I understand that if I have two models A and B and A is nested in B then, given some data, I can fit the parameters of A and B using MLE and apply the generalized log likelihood ratio test. In particular, the distribution of the test should be $\chi^2$ with $n$ degrees of freedom where $n$ is the difference in the number of parameters that $A$ and $B$ have.

However, what happens if $A$ and $B$ have the same number of parameters but the models are not nested? That is they are simply different models. Is there any way to apply a likelihood ratio test or can one do something else?

$\endgroup$
9
$\begingroup$

The paper Vuong, Q. H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica, 307-333. has the full theoretical treatment and test procedures. It distinguishes between three situations, "Strictly Non-nested Models", "Overlapping Models", "Nested Models", and also examines cases of misspecification. It is therefore no-accident that it finds that for some cases, the test statistic is distributed as a linear combination of chi-squares.

The paper is not light, neither it proposes an "off-the-shelf" testing procedure. But, for once, its (close to) 3,000 citations speak of its merits, being an inspired combination of classical testing framework and the information-theoretic approach.

$\endgroup$
1
$\begingroup$

The Generalised likelihood ratio test DOES NOT work the way you are saying. See for example the following lecture notes:

http://www.maths.manchester.ac.uk/~peterf/MATH38062/MATH38062%20GLRT.pdf

http://www.maths.qmul.ac.uk/~bb/MS_Lectures_12b.pdf

The GLRT is defined for hypothesis of the type:

$$H_0: \theta\in\Theta_0 \,\,\, vs. \,\,\, H_1: \theta\in\Theta_1,$$

where $\Theta_0\cap\Theta_1=\emptyset$ and $\Theta_0\cup\Theta_1=\Theta$.

For the framework you describe, you can compare the models using other tools such as AIC and BIC. Also Bayes factors, if you are willing to go full Bayesian.

$\endgroup$
  • $\begingroup$ Welcome to CV. Perhaps it would be of interest to you to look up the paper I am mentioning in my own answer to this question. $\endgroup$ – Alecos Papadopoulos Jul 30 '14 at 21:33
  • $\begingroup$ @AlecosPapadopoulos Thank you for the reference. I took a quick glance and, as expected, the conditions for that sort of GLRT to work are very very (very very) restrictive. So, I would prefer to go for something safer. I know it is highly cited, apologies for the blasphemy. $\endgroup$ – Waterman Jul 30 '14 at 21:35
  • $\begingroup$ @AlecosPapadopoulos In particular, I find the compactness of the parameter space condition (Assumption A2) extremely unappealing. $\endgroup$ – Waterman Jul 30 '14 at 21:40
  • $\begingroup$ The very instructive (although probably not real) historical anecdote around Laplace's magnum opus is that Napoleon the Great read it and commented to Laplace "I see you do not mention God anywhere in your book", to which Laplace supposedly replied "I did not need that hypothesis"... meaning that the concept of "sacred" is not needed in science, and so, there can be no blasphemy. $\endgroup$ – Alecos Papadopoulos Jul 30 '14 at 21:41
  • 1
    $\begingroup$ ...as for your second comment on Assumption A2, I guess it means that the whole maximum likelihood framework does not quite meet the needs of your field, except perhaps when the distributions involved have log-concave densities. $\endgroup$ – Alecos Papadopoulos Jul 30 '14 at 22:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.