6
$\begingroup$

Do I understand this correctly:

You fit a model with unknown parameters to a dataset. You choose the parameters so the likelihood of the dataset under the model is maximal. Let this be $L_{max, model1}$. You do the same with a more advanced model. Again you pick the coefficients of the model so that the likelihood of the dataset under the model is maximal. Let this be $L_{max, model2}$

To see which model is the best, you compare the two (maximum) likelihoods. Now what confuses me is the definition of the likelihood ratio test:

"The likelihood ratio is the ratio of the likelihoods of the two models... ".

Shouldn't that be "The likelihood ratio is the ratio of the maximum likelihoods of the two models..."?

$\endgroup$

2 Answers 2

5
$\begingroup$

You're right in the context of "submodel" testing, the likelihood ratio statistic is the ratio of the maximum likelihoods (not the maximum likelihood estimates: the maximal values of the likelihoods). Consider a statistical model with likelihood $l(\theta \mid y)$ where $y$ is the vector of observations generated from a distribution with parameter $\theta$ belonging to some space $\Theta$. Let $\Theta_0 \subset \Theta$ and imagine you are interested in testing $H_0\colon\{\theta \in \Theta_0\}$. The likelihood ratio statistic is $$lr(y) = \frac{\sup_{\theta \in \Theta}l(\theta \mid y)}{\sup_{\theta \in \Theta_0}l(\theta \mid y)}. $$

But when the test hypotheses are $H_0 \colon\{\theta =\theta_0\}$ vs $H_1 \colon\{\theta =\theta_1\}$, as in the classical Neymann-Pearson lemma, then the likelihood ratio statistic is the ratio of the likelihoods: $$lr(y) = \frac{l(\theta_1 \mid y)}{l(\theta_0 \mid y)}.$$

$\endgroup$
7
  • $\begingroup$ Is this the distinction between a generalized likelihood ratio test for a composite hypothesis (first case) vs. a likelihood ratio test for a simple hypothesis (second case) as in the answer from @Scortchi ? $\endgroup$
    – Jon
    Commented Jan 20, 2014 at 14:13
  • $\begingroup$ @psychometriko I don't know and I'd like to know that too. $\endgroup$ Commented Jan 20, 2014 at 14:16
  • $\begingroup$ My mathematical stats isn't the strongest, but that's what I suspected when reading your (very elegantly worded, for a complicated topic) answer. Hopefully someone can come and shed light on this! $\endgroup$
    – Jon
    Commented Jan 20, 2014 at 14:20
  • $\begingroup$ Thank you for your answer! Just for my understanding, can you please explain what "sup" means? $\endgroup$
    – Kasper
    Commented Jan 20, 2014 at 15:24
  • 1
    $\begingroup$ @Kasper $\sup_{\theta \in \Theta}\ldots$ means supremum (maximum) when $\theta$ runs over $\Theta$. $\endgroup$ Commented Jan 20, 2014 at 15:41
2
$\begingroup$

The distinction between the likelihood ratio for completely specified probability mass or density functions (simple hypotheses) & the likelihood ratio for incompletely specified ones (composite hypotheses) is sometimes expressed by calling the latter a generalized likelihood ratio. So your quote could be giving a precise definition of the likelihood ratio proper, or an vague definition (because it doesn't specify how particular parameter values are picked to calculate the likelihood for each model) of the generalized likelihood ratio.

$\endgroup$
2
  • $\begingroup$ I am sorry but I don't get this... $\endgroup$
    – Kasper
    Commented Jan 20, 2014 at 15:51
  • $\begingroup$ @Stéphane's first formula is what's sometimes called the generalized likelihood ratio. $\endgroup$ Commented Jan 20, 2014 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.