Likelihood ratio test for composite alternative hypothesis

Question

I must admit that my general understanding on how to create critical region to test hypothesis against composite alternative hypothesis is still shaky. Therefore please pardon if my question is too trivial.

Lets take one example from Likelihood ratio test to determine if average number of accidents has dropped?

Here $H_1: \lambda < 4.$

It is mentioned that general form of likelihood ratio test takes the form as

$$\Lambda(x)=\frac{\sup_{\lambda\in\Theta_{H_{0}}}L(\lambda|x)}{\sup_{\lambda\in\Theta}L(\lambda|x)} < c. $$

I think I understand why this is so, as in order to reject the null, I must have higher likelihood under the alternate hypothesis given a fixed sample. Therefore numerator must be smaller to the denominator.

After following some direct mathematics final critical region comes as $\sum x<c$

However it appears that in this entire derivation we did not take the direction of $H_1$ which is $H_1: \lambda < 4.$

How the critical region would change if I had $H_1: \lambda > 4$ or $H_1: \lambda \neq 4$?

Answer 1

Firstly, you've got to be careful when you write something like

$$ X_1, \ldots, X_n \sim \mathrm{Exp}(\lambda) $$

because there are two common parameterisations of the Exponential distribution. In the example you linked to, $\lambda$ is the scale ("Where $\lambda$ is the population mean time between accidents...", i.e. $\mathbb{E}(X_1)=\lambda$), whereas in the other answer posted to your question $\lambda$ is assumed to be the rate (so $\mathbb{E}(X_1)=1/\lambda$). The same error appears in the answer to the linked question. The correct likelihood for the scale parameterisation is

$$ L(\lambda;\underline{x}) = \prod_{i=1}^n f_{X_i}(x_i;\lambda) = \prod_{i=1}^n \frac{1}{\lambda}e^{-x_i/\lambda} = \lambda^{-n} e^{-n\bar{x}/\lambda} $$ and the MLE is $\hat{\lambda}=\bar{X}$, i.e. we use the sample mean to estimate the population mean.

Secondly, the likelihood-ratio test is suitable when we're testing hypotheses of the form $H_0: \, \theta \in \Theta_0$ versus $H_1: \, \theta \in \Theta_0^c$, the complement of $\Theta_0$. In your example, if we want $H_1: \, \lambda < 4$, then we have to take $H_0: \, \lambda \geq 4$.

In the denominator of $\Lambda(x)$ you're maximising the likelihood over the whole of $\Theta$, so the estimator is $\hat{\lambda}=\bar{X}$ (the unconstrained MLE).

Now consider the numerator of $\Lambda(x)$. You want to maximise the likelihood over $\Theta_{H_0}$ to obtain $\hat\lambda_0$ (the constrained MLE). There are two cases:

1. If it so happens that $\bar{x} \geq 4$, the constrained and the unconstrained MLE are the same, so $\Lambda(x)=1$. We clearly don't want to reject $H_0$, as there is no evidence in favour of $H_1$.

2. If $\bar{x} < 4$, there is some evidence in favour of $H_1$. The constrained MLE is then $\hat{\lambda}_0=4$ (can you see why?), so we get

$$ \Lambda(x) = \frac{\hat\lambda_0^{-n} e^{- n \bar{x}/\hat\lambda_0}}{\hat\lambda^{-n} e^{- n \bar{x}/\hat\lambda}} = \frac{4^{-n} e^{- n \bar{x}/4}}{\bar{x}^{-n} e^{- n \bar{x}/\bar{x}}} \propto \bar{x}^n e^{-n\bar{x}/4} $$ We can easily verify that this is a increasing function of $\bar{x}$ when $\bar{x}<4$ (check the derivative). We reject $H_0$ when $\Lambda(x)$ is small, which corresponds to low values of $\bar{x}$.

---

If we wanted to test $H_0: \, \lambda \leq 4$ versus $H_1: \, \lambda > 4$, the calculations are very similar. We care about the case where $\bar{x}>4 $, and you can check that the likelihood ratio is a decreasing function of $\bar{x}$ in this region, i.e. we reject $H_0$ for high values of $\bar{x}$.

Finally, testing $H_0: \, \lambda = 4$ versus $H_1: \, \lambda \neq 4$ is a little trickier, because $\Lambda(x)$ is no longer monotonic in $\bar{x}$ (or any other function of the sample), i.e. we want to reject $H_0$ for very high or very low values of $\bar{x}$. We would take $-2\log\Lambda(x)$ and use the fact that this is approximately Chi-square distributed under $H_0$.

Answer 2

Here $\Theta_0:=\{\lambda: \lambda =4\}$ and $\Theta:=\{\lambda:\lambda\leq 4\}.$

Now \begin{align}\frac{\mathrm d}{\mathrm d\lambda}\mathcal L(\lambda|\mathbf x) &=\frac{\mathrm d}{\mathrm d\lambda}\left[-n\ln \lambda-\frac{ n\bar { x}}{\lambda}\right]\\ &= -\frac n\lambda+\frac{n\bar{ x}}{\lambda^2}.\tag 1\label 1\end{align} From $\eqref 1,$ $$\frac{\mathrm d}{\mathrm d\lambda}\mathcal L(\lambda|\mathbf x) > 0\implies \lambda <\bar x .\tag 2\label 2$$

If ${\bar x}\in(0, 4],$ then from $\eqref 2,~\hat{\lambda}_{\text{MLE}}={\bar x}.$ If ${\bar x}\in(4, \infty),$ then $\hat{\lambda}_{\text{MLE}} =4.$

So, if ${\bar x}\in(4, \infty),$ $$\ell(\mathbf x) =1\tag{ 3.I}\label 3$$ and if ${\bar x}\in(0, 4],$ then $$\ell(\mathbf x) =\frac{4^{-n}\exp\left(-\frac{n\bar x}{4}\right)}{(1/\bar x)^n\exp(-n)}.\tag{3.II}$$

$\eqref 3$ is trivial.

Observe $$\frac{\mathrm d}{\mathrm d\bar x}\ell(\mathbf x) =k\times \left[n{\bar x}^{n-1}\exp\left(\frac{-n\bar x}{4}\right)\left(1-\frac{\bar x}4\right)\right].$$

So, when ${\bar x}\in(0, 4],$

$$\ell(\mathbf x) < \textrm{some constant}\iff \bar x< \textrm{some other constant}.$$

The test function then follows.

Similarly, one can proceed along the same approach for $\mathcal H_1: \lambda > 4.$