I have a small confusion over describing the cutoff point for the critical region in a likelihood ratio test when the null hypothesis is composite. Take this exercise in particular:

Let $(X_1,X_2,\ldots,X_n)$ be a random sample from a shifted exponential distribution with density $$f_{\theta}(x)=e^{-(x-\theta)}\mathbf1_{x\ge\theta}\quad,\,\theta\in\mathbb R$$

I am to derive the likelihood ratio test of size $\alpha$ for testing $$H_0:\theta\le \theta_0\quad\text{ against }\quad H_1:\theta>\theta_0,$$ where $\theta_0$ is a specified value of $\theta$.

Given the sample $(x_1,x_2,\ldots,x_n)$, the likelihood function is

$$L(\theta\mid x_1,\ldots,x_n)=\prod_{i=1}^n f_{\theta}(x_i)=\exp\left[-\sum_{i=1}^n (x_i-\theta)\right]\mathbf1_{x_{(1)}\ge\theta}\quad,\,\theta\in\mathbb R$$

Unrestricted MLE of $\theta$ is clearly $$\hat\theta=X_{(1)}$$

And I think the restricted MLE of $\theta$ when $\theta\le \theta_0$ is $$\hat{\hat\theta}=\begin{cases}\hat\theta&,\text{ if }\hat\theta\le\theta_0 \\ \theta_0&,\text{ if }\hat\theta>\theta_0 \end{cases}$$

So my LR test statistic is

\begin{align} \Lambda(x_1,\ldots,x_n)&=\frac{\sup_{\theta\le\theta_0}L(\theta\mid x_1,\ldots,x_n)}{\sup_{\theta}L(\theta\mid x_1,\ldots,x_n)} \\\\&=\frac{L(\hat{\hat\theta}\mid x_1,\ldots,x_n)}{L(\hat\theta\mid x_1,\ldots,x_n)} \\\\&=\begin{cases}1&,\text{ if }\hat\theta\le\theta_0\\\\\frac{L(\theta_0\mid x_1,\ldots,x_n)}{L(\hat\theta\mid x_1,\ldots,x_n)}&,\text{ if }\hat\theta>\theta_0\end{cases} \end{align}

If $\hat\theta\le\theta_0$, we trivially accept $H_0$.

Now when $\hat\theta>\theta_0$,

$$\Lambda(x_1,\ldots,x_n)=\frac{L(\theta_0\mid x_1,\ldots,x_n)}{L(\hat\theta\mid x_1,\ldots,x_n)}=e^{n(\theta_0-x_{(1)})}$$

Therefore, when $x_{(1)}>\theta_0$,

\begin{align} \Lambda(x_1,\ldots,x_n)<\text{ constant }&\implies x_{(1)}-\theta_0>\text{ constant } \end{align}

My confusion is whether I should rewrite that last line as $x_{(1)}>\text{ constant }$ or keep it as it is.

We know that an appropriate test statistic here is $$2n(X_{(1)}-\theta)\sim \chi^2_2$$

But under $H_0$, how is $\theta<\theta_0$ reflected in this statistic (if it was a simple null it would have been fine)? Is it correct to say, that under $H_0$, $2n(X_{(1)}-\theta_0)\sim \chi^2_2$?

If I describe the critical region as $x_{(1)}>c$, then I have to find $c$ subject to $$P_{H_0}(X_{(1)}>c)=\alpha$$

I can do this by finding the probability directly. But suppose I want to find the cutoff point in terms of the $\chi^2_2$ fractile. Then I get for some $k (=c-\theta_0)$, $$P_{H_0}(2n(X_{(1)}-\theta_0)>2n k)=\alpha,$$ which gives me $$2nk=\chi^2_{2;\alpha}\implies k=\frac{1}{2n}\chi^2_{2;\alpha},$$ where $\chi^2_{2,\alpha}$ is the $(1-\alpha)$th fractile of a $\chi^2_2$ distribution.

Then my decision rule for $\hat\theta>\theta_0$ would be "Reject $H_0$ at size $\alpha$ if $X_{(1)}>\frac{1}{2n}\chi^2_{2;\alpha}$".

Is there anything wrong with the last line? Or should I write the cutoff point in terms of $c$ , that is $c=\frac{1}{2n}\chi^2_{2;\alpha}+\theta_0$ ?

In other words, should I write the cutoff point including or excluding $\theta_0$?

This question arises because I have been told that the LRT coincides with the UMP test whenever the latter exists. And for the UMP test (UMP test exists because $f_{\theta}$ has the MLR property), the cutoff point for the critical region is expressed in terms of $\theta_0$.


1 Answer 1

  • You are not gonna be able to define a $c$ such that $P_{H_0}(X_{(1)}>c)=\alpha$ when $H_0$ is not a simple hypothesis. The best you could do is find the smallest c such that $P_{H_0}(X_{(1)}>c)<\alpha$

    The composite $H_0: \theta \leq \theta_0$ is too ambiguous in order for you to form a conditional probability statement. Namely: for two different $\theta_a, \theta_b \in (-\infty,\theta_0]$ you will have different distributions of $X_{(1)}$.

    So you will have to use $$P(X_{1}>c|\theta_a) < P(X_{1}>c|\theta_b) \qquad \text{ if } \theta_a < \theta_b$$ and if you find $c$ such that $$P(X_{1}>c|\theta_0) = \alpha$$ then for any $\theta \leq \theta_0$ you will have $$P(X_{1}>c|\theta \leq \theta_0) \leq \alpha$$.

    But the equality is not general and only holds when $\theta = \theta_0$ so the best you can do is $P(X_{1}>c|H_0) \leq \alpha$ and not $P(X_{1}>c|H_0) = \alpha$

  • The $\theta_0$ should be included in your cutoff point (otherwise your test/cutoff would be independent from $\theta_0$ which makes no sense).

    If you plot it, compute it, test it, experiment with it, then you should be able to find out the correct expression.

    Basically you want to test for a high value so: $2n(X_{(1)}-\theta_0)\sim \chi_2^2$ means you look for $2n(x_{(1)}-\theta_0) > \chi_{2;\alpha}^2$ and you only need to rewrite this in a convenient form (your questions and confusion seems to be about what this convenient form is, but from the mathematical point of view it is arbitrary; $x_{(1)}>\frac{1}{2n}\chi^2_{2;\alpha}+\theta_0$ and $x_{(1)} - \theta_0>\frac{1}{2n}\chi^2_{2;\alpha}$ are equivalent).

  • $\begingroup$ I agree with the second bullet that I should directly say $x_{(1)}-\theta_0>\text{ constant }\implies 2n(x_{(1)}-\theta_0)>\chi^2_{2;\alpha}$. That is, $x_{(1)}>\frac{1}{2n}\chi^2_{2;\alpha}+\theta_0$. The confusion was whether I should push $\theta_0$ to the rhs of $x_{(1)}-\theta_0>\text{ const. }$ since $\theta_0$ is also constant. But could you elaborate on the first bullet? $\endgroup$ Nov 22, 2018 at 20:01
  • $\begingroup$ Say I am given $\theta_0=1$ explicitly. Then $x_{(1)}-1>\text{ const. }$ implies $x_{(1)}>\text{ const. }$ and then the cutoff becomes independent of the $\theta_0$ as I had absorbed the $1$ as a constant. That was the doubt. (+1) Nonetheless. $\endgroup$ Nov 23, 2018 at 17:55
  • $\begingroup$ The constant (and also the cutoff) will become dependent of $\theta_0$, namely $x_{(1)}> constant = \frac{1}{2n}\chi^2_{2;\alpha}+\theta_0$ $\endgroup$ Nov 23, 2018 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.