I'm having trouble defining the reject region based on the generalized likelihood ratio test. This is from a question of past exam I'm self-studying and still have the doubt, given that I got it wrong.

Let $X_{1},\ldots,X_{n}$ a sample from an exponential distribution $X \sim \text{Exp}(\theta)$ with $\theta = \mathbb{E}[X]$. The parameter space is $\Omega = \{\theta \mid\theta>0\}.$

I need to contrast the next hypothesis: $H_{0}:\theta \geq \theta_{0}$ vs $H_{1}: \theta< \theta_{0}$.

The set of the null hypothesis is: $\Omega_{0} = \{\theta \mid\theta \geq \theta_{0}\}$ and the set of the alternative hypothesis is: $\Omega_{1} = \{\theta \mid\theta < \theta_{0}\}$. So that $\Omega = \Omega_{0} \cup\Omega_{1}.$

The likelihood function for $\theta$ is given by: $$ L(\theta) = \theta^{-n} \exp\{ -\theta^{-1} \bar{X} \} $$

I know the generalized likelihood ratio is given by $$\Lambda = \frac{\sup_{\theta \in \Omega_{0}} L (\theta)}{\sup_{\theta \in \Omega} L (\theta)}$$

and serves as a test statistic. For the denominator, we have that the MLE of $\theta$ is given by $\hat{\theta} = \bar{X}$, and with this the denominator is $$L(\hat{\theta}) = \bar{X}^{-n}\exp\{ -\bar{X}^{-1}n\bar{X} \} = \bar{X}^{-n}\exp\{ -n \}$$

My problem is: how do I solve for the numerator $\sup_{\theta \in \Omega_{0}} L (\theta)$ ? Because I already know how to solve the test given a simple null hypothesis $H_{0}: \theta = \theta_{0}$ and I'm really stucked with this problem.

Thank you for your help.

  • $\begingroup$ See these notes Exmp 4.11 p78, where the same problem is discussed. There $\theta$ is the exponential rate, not mean. $\endgroup$
    – BruceET
    May 30, 2019 at 21:08

2 Answers 2


Unrestricted MLE of $\theta$ is as you say $\hat\theta=\overline X$, the sample mean.

Now under the restriction $\theta\ge\theta_0$, argue that MLE of $\theta$ must be $$\hat{\hat\theta}=\begin{cases}\hat\theta&,\text{ if }\hat\theta\ge\theta_0 \\ \theta_0&,\text{ if }\hat\theta<\theta_0\end{cases}$$

So depending upon whether $\overline X\ge \theta_0$ or $\overline X<\theta_0$, the likelihood ratio statistic takes the form

\begin{align} \Lambda=\frac{\sup_{\theta\ge\theta_0} L(\theta)}{\sup_{\theta}L(\theta)}&=\frac{L(\hat{\hat\theta})}{L(\hat\theta)} \\&=\begin{cases}1&,\text{ if }\hat\theta\ge\theta_0 \\\\ \frac{L(\theta_0)}{L(\hat\theta)}&,\text{ if }\hat\theta<\theta_0\end{cases} \end{align}

Now it is a matter of studying this ratio as a function of $\overline X$ when $\hat\theta<\theta_0$. Remember to reject $H_0$ for small values of $\Lambda$. The case corresponding to $\hat\theta\ge\theta_0$ leads to trivial acceptance of $H_0$.

  • $\begingroup$ Thank you @StubbornAtom. Your explanation was very clear. $\endgroup$ May 31, 2019 at 0:51

Test. Once you figure out that you want to reject for small $\bar X,$ the task remains to find the critical value $c$ for a test of $H_0: \theta \ge \theta_0$ vs $H_a: \theta < \theta_0$ at level $\alpha = 0.05.$ That is, $P(\bar X \ge c\,|\,\theta_0) = 0.05.$

Null distribution Suppose that $\bar X$ is the mean $\bar X$ of a random sample $X_1, X_2, \dots, X_n$ from $\mathsf{Exp}(\theta_0),$ so that $E(X_i) = \theta.$ Then you can use moment generating functions to show $\bar X \sim \mathsf{Gamma}(\text{shape} = n, \text{scale}=\theta/n).$ Thus $E(\bar X) = \theta$ and $SD(\bar X) = \theta/\sqrt{n}).$

The following simulation in R of a million samples of size $n =25$ from $\mathsf{Exp}(\text{mean} = \theta = 50)$ $\equiv$ $\mathsf{Exp}(\text{rate} = \lambda = 0.02)$ compares a histogram of the simulated distribution of means $\bar X$ with the density function of $\mathsf{Gamma}(n, \text{scale} = \theta/n) \equiv \mathsf{Gamma}(n, \text{rate} = n/\theta).$ (Perhaps see Wikipedia on gamma distributions; R uses the rate parameter.)

n = 25;  th = 50;  lam = 1/th
a = replicate( 10^6, mean(rexp(n, lam)) )
lbl = "Simulated Sums of Exponential Data with Gamma Density"
hist(a, prob=T, br = 50, col="skyblue2", main = lbl)
  curve(dgamma(x, n, n/th), 0, 120, add=T, lwd=2)

enter image description here

Critical value. Then, testing $H_0: \theta \ge \theta_0 = 50$ vs. $H_a: \theta < 50,$ based on $n = 25$ exponential observations, we can use R to find the critical value as $c = 34.764.$ That is, $P(\bar X \le 34.764\, |\, \theta_0 = 50) = 0.05.$

th.0 = 50;  n = 25
c = qgamma(.05, n, n/th.0);  c
[1] 34.76425

Power. If we happen to sample from an exponential distribution with mean $\theta_a = 40$ (rate $\lambda_a = 0.025),$ what is the probability of rejecting $H_0?$ The result from R is only $P(\bar X \le 34.764\, |\, \theta_a = 40) = 0.268.$ So using this test, $n = 25$ observations is not enough reliably to discover if the true population mean is $\theta_a = 40$ instead of $\theta_0 = 50.$ However, the power against the alternative value $\theta_a = 30$ is almost 80%.

n = 25; th = 40; c = 34.764
pwr = pgamma(c, n, n/th);  pwr
[1] 0.2682961

n = 25; th = 30; c = 34.764
pwr = pgamma(c, n, n/th);  pwr
[1] 0.7942828
  • $\begingroup$ Thank you @BruceET. Your answer was tremendously useful and super clear. $\endgroup$ May 31, 2019 at 0:58
  • $\begingroup$ Glad it was helpful. $\endgroup$
    – BruceET
    May 31, 2019 at 1:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.