2
$\begingroup$

I am preparing for an exam, I've been reading through Likelihood Ratio Test, but don't get it.

Example 8.2.3 (page 376 in "Statistical Inference" Casella and Berg)

Let ${{x}_{1}},...{{x}_{n}}$ be a random sample from an exponential population with pdf $$f(x|\theta )=\left\{ \begin{array}{*{35}{l}} {{e}^{-(x-\theta )}} & x\ge \theta \\ 0 & x<\theta \\ \end{array} \right.$$ ${{H}_{0}}_{{}}:\theta \le \theta $ versus ${{H}_{1}}:\theta >{{\theta }_{0}}$

The Likelihood function is, $$L(\theta |x)=\left\{ \begin{array}{*{35}{l}} {{e}^{-\Sigma {{x}_{i}}+n\theta }} & \theta \ge {{x}_{(1)}} \\ 0 & \theta <{{x}_{(1)}} \\ \end{array} \right.$$ The book says: Clearly, $L(\theta |x)$ is an increasing function of $\theta$ on $-\infty < \theta \le {{x}_{(1)}}$. (Does this even matter?). Thus, the denominator of $\lambda(x)$, the unrestricted maximum of $L(\theta|x)$, is

$$L({{x}_{1}}|x)={{e}^{-\Sigma{{x}_{i}}+n{{x}_{(1)}}}}$$ If ${x}_{1}\le{\theta}_{0}$, the numerator of $\lambda(x)$ is also $L({x}_{(1)}|x)$. But since we are maximizing $L(\theta|x)$ over $\theta\le{\theta}_{0}$, the numerator of $\lambda(x)$ is $L({\theta}_{0})$ if ${x}_{1}>{\theta}_{0}$. Therefore, the likelihood ratio test statistic is $$\lambda (x)=\left\{ \begin{array}{*{35}{l}} 1 & {{x}_{(1)}}\ge {{\theta }_{0}} \\ {{e}^{-n\left( {{x}_{(1)}}-{{\theta }_{0}} \right)}} & {{x}_{(1)}}<{{\theta }_{0}} \\ \end{array} \right.$$

Rejection region $$\{x:{{x}_{(1)}}\ge {{\theta }_{0}}-\frac{\log c}{n}\}$$

I read through the whole thing, it was long and tedious to follow, I'm quite lost. Can someone else explain this? Maybe with a better/simple toy example? I want to understand this, not just follow through!

Is this a simple concept, I am over-complicating it?

$\endgroup$
1
  • $\begingroup$ I think that the $L(\theta \mid x)$ should be positive for $x_(1) \geq \theta$, rather than the other way around. $\endgroup$
    – Charlie
    Commented May 2, 2012 at 17:54

2 Answers 2

1
$\begingroup$

Clearly, L(θ|x) is an increasing function of θ on −∞<θ≤x1 . (Does this even matter?).

Yes, this matters. Since L() is increasing wrt θ, we want to choose θ as large as we can in order maximize L(), but we have the constraint that θ≤x1, so the largest we can choose θ is x1.

So x1 is our unrestricted maximum likelihood estimator for θ.

In the numerator, however, we have the additional constraint θ≤θ0, so we have to take that into consideration, too.

Two things can happen.

1) We have x1<θ0 and the constraint doesn't matter, so our unrestricted MLE is the same as our restricted MLE, so the likelihoods are the same, and thus we have a likelihood ratio of 1.

2) We have x1>θ0, so we can't choose x1 without violating θ≤θ0. We've already seen that L() is an increasing function wrt θ, so choosing θ as large as we can will maximize L, and in this case, this corresponds to θ=θ0. When we take the ratio of the two likelihoods, we end up with the second term.

The rejection region will be found by inverting the test statistic.

Hope this helps.

To give a little more concrete and simple example, consider:

I flip a coin 10 times and observe 8 heads. I would like to know if the coin is fair.

This can be characterized as a binomial(10,p) distribution, and we are testing H0: p=0.5 vs p not equal 0.5.

In the numerator, I'll plug in p=0.5 to the binomial distribution, since that's p0.

In the denominator, I'll plug in p=0.8 to the same binomial distribution, since that's the MLE.

I'll reject the null hypothesis when this ratio is very small, because the null hypothesis is very unlikely compared to the alternative hypothesis.

$\endgroup$
5
  • $\begingroup$ I somewhat get it now. We want to minimize the negative exponent, so we can maximize the likelihood. The requirement is $\theta \le x$, so we pick the smallest x possible, x1 in our case. Am I anything near to be correct? $\endgroup$ Commented Apr 2, 2012 at 19:30
  • $\begingroup$ In some sense, yes, but you should be viewing this as choosing the largest θ rather than the smallest x, as the data (x) are given, and we are maximizing with respect to the parameter (θ) $\endgroup$
    – Gschneider
    Commented Apr 2, 2012 at 19:45
  • $\begingroup$ Thanks, that's a great point, I need to correct the way I view things. $\endgroup$ Commented Apr 2, 2012 at 20:34
  • $\begingroup$ Just worked through to your example. Your p is the the parameter? p0 is an observed value. I can confused when assigning p or p0, and you have 0.5 and 0.8. Could you explain? $\endgroup$ Commented Apr 2, 2012 at 23:44
  • $\begingroup$ P is the parameter, an unobservable probability. P0 is the probability under the null hypothesis, 0.5 (50-50 chance of heads). The unconstrained MLE is 0.8 or 8/10, the usual MLE for the binomial distribution. $\endgroup$
    – Gschneider
    Commented Apr 3, 2012 at 0:12
0
$\begingroup$

I do not follow your example but I can tell you how I was taught the Likelihood ratio and I'm sure this will be more understandable.

A Likelihood ratio test can be used to compare two nested models (this is important). Where the one model is a special case of the other model. As usual the null is the most restrictive case and expresses the baseline model described below. The test can be used whether adding a few parameters increase the fit (expressed by the log likelihood or -2 log likelihood criterium) of the model. For example, you have a model with 2 X'es and you want to know whether a model which has those 2 X'es and 2 additional X'es performs better, statistically. Adding more parameters in your model always improves the log likelihood but you want to know whether this improvement is large enough to be statistically significant.

The Likelihood ratio test uses the difference between the -2 log likelihoods of the base model (here, the model with 2 X'es) and the extended model (the model with 4 X'es). This test statistic is then compared with the Chi-square distribution with DF the number of X'es added.

The models ofcourse must make use of the log likelihood criteria (logit, probit models for example).I think what L(θ|x) means in your example, is the Likelihood function.

I'm by no means very theoretical, more a practitioner but I hope this helps.

More information can be found on Google ie. http://en.wikipedia.org/wiki/Likelihood-ratio_test

$\endgroup$
3
  • $\begingroup$ Maybe I wasn't clear, I edited it. I am working with hypothesis testing, are you talking about regression models? Thanks for your answer. $\endgroup$ Commented Apr 1, 2012 at 22:31
  • $\begingroup$ Yes I am talking about regression models. I don't think my comment is appropriate. Maybe the Wikipedia link may shed some more light on your question? $\endgroup$
    – Raynor
    Commented Apr 1, 2012 at 22:50
  • $\begingroup$ yes, Wikipedia definitely gets me started. I have been looking around for references, but these stuff doesn't seem that popular/prevalent on the web. I was expecting to see some very well-written pdf, but no luck. A little surprising. $\endgroup$ Commented Apr 2, 2012 at 2:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.