2
$\begingroup$

Suppose that we are given an exponential distribution model with a pdf $f(x,\theta) = \theta^{-1}\exp(-x/\theta)$ with an iid sample $X_1, ..., X_n$, and we would like to test hypothesis $H_0 : \theta = 1$ and $H_1 : \theta > 1$. We shall investigate the limiting distribution of the likelihood test statistic $2n(l_n (\hat\theta) - l_n (1))$ where $l_n(\theta)$ is the log-likelihood function $\sum \log f(x,\theta$) and $\hat \theta$ is the MLE of $\theta$ in the parameter space. We shall investigate the asymptotic behavior of the test statistic.

Note that the parameter space is not the whole real line, so it does not create an open neighborhood around $H_0: \theta = 1$). Here is what I have discovered so far: we know that the MLE of exponential distribution is $\overline X_n$, the sample mean if we are allowed to take the whole parameter space. Using a similar argument, if the sample mean is greater than one, the same argument works. However, if it is smaller than or equal to one, the MLE is 1 (we can show it using a monotonicity argument). Hence, the test statistic becomes

$\begin{cases} 2n(\ln \overline X_n - (\overline X_n - 1)) && \text{if $\overline X_n > 1$} \\ 0 && \text{if $\overline X_n \leq 1$} \end{cases}$

we can use then a taylor expansion for the first case to show that it is indeed approximately equal to $n(\overline X_n - 1)^2$ that has a $\chi^2$-distribution asymptotically. However, how can we incorporate the case where $\overline X_n \leq 1$? If $H_1$ were $\theta \neq 1$ it would have been safe and sound. I guess the test becomes $\sum \overline X_n > k$ for some $k$, due to the zero part for the second case?

$\endgroup$
  • $\begingroup$ Why not allow $\theta \gt 0$ and test $H_0: \theta \le 1$? All these complications then disappear. $\endgroup$ – whuber Feb 21 '16 at 21:41
  • $\begingroup$ @whuber MLE will be $2n(\ln \overline X_n - (\overline X_n - 1))$ for all $\overline X_n$, but it still remains to show the asymptotic behavior of it.. and if that's the case, the taylor expansion argument would not work as $\overline X_n$ does not necessarily converge to 1. $\endgroup$ – Dave Feb 22 '16 at 3:55
1
$\begingroup$

As @whuber pointed out, the most practical solution would be to change how you define your test. However, if there is some reason why you would prefer not to do so, I explain what would happen below.

To start with my conclusions, I find that for most any reasonable choice of significance level, you can obtain a p-value for your hypothesis via a one-tailed "z-test" using the asymptotic normal distribution of $\bar X$ under the null (which is given by the central limit theorem).

To restate the problem, let $X_1,...,X_n \stackrel{iid}{\sim} exp(\theta)$ and consider the hypotheses $$ H_0:\; \theta=1\;\;\;H_1 \theta > 1 $$

So the likelihood is $$L(\theta|\mathbf{X})=\theta^{-1}\exp(\theta^{-1}\sum_{i=1}^n X_i)$$ and the corresponding likelihood ratio test statistic is $$ \lambda=-2\ln\bigg[\frac{\sup\{\,L(\theta\mid \mathbf{X}):\theta\in\Theta_0\,\}}{\sup\{\,L(\theta\mid \mathbf{X}):\theta\in\Theta\,\}}\bigg]= -2\ln\bigg[\frac{L(\theta=1 \mid \mathbf{X})}{\sup\{\,L(\theta\mid \mathbf{X}):\theta\ge 1\,\}}\bigg] = $$ $$ -2\ln\bigg[\frac{L(\theta=1 \mid \mathbf{X})}{L(\theta=\max(\bar X, 1) \mid \mathbf{X})}\bigg] $$

Notice that my maximum likelihood estimate of $\theta$ is $\hat \theta = \max(\bar X,1)$ as opposed to just $\bar X$. This is because when you define the null and alternative hypothesis you are assuming that

$$ \Theta = \Theta_0 \cap \Theta_1 = \{1\} \cap (1,\infty)=[1,\infty) $$ In layman's terms, it is impossible by your definition that $\theta<1$, so it would not make sense to use $\bar X < 1$ as an estimate for $\theta$. In the case that $\bar X< 1$, the estimate which yields the highest likelihood, but still resides in the parameter space, is $\hat \theta=1$

Now the boundary problem issue you are referring to is well defined. Under the null, $\lambda$ will not converge to a chi-squared distribution. This problem is well researched. In my answer here I provide many sources which discuss the problem.

In this specif case, it can be shown (using the central limit theorem) that the asymptotic distribution of $\hat \theta$ is a censored normal distribution;

$$ p(\hat \theta \mid \theta=1)=\begin{cases} \phi(\hat \theta \mid \mu=1,\sigma=1/\sqrt{n}) \;\;\mathrm{if}\;\;\hat \theta > 1 \\ 0.5\times \delta(\hat \theta-1)\;\;\mathrm{if}\;\;\hat \theta=1 \\ 0\;\;\mathrm{otherwise}\end{cases} $$ where $\phi(x \mid \mu,\sigma)=\frac{1}{\sigma\sqrt{2\pi}}\exp\bigg(-\frac{(x-\mu)^2}{2\sigma^2} \bigg)$ is the Gaussian pdf, and $\delta(x)$ is the Dirac delta function. The limiting CDF is then;

$$ P(\hat \Theta \le \hat \theta \mid \theta=1)=\begin{cases} \Phi(\hat \Theta \le \hat \theta \mid \mu=1,\sigma=1/\sqrt{n}) \;\;\mathrm{if}\;\;\hat \theta > 1 \\ 0.5\;\;\mathrm{if}\;\;\hat \theta=1 \\ 0\;\;\mathrm{otherwise}\end{cases} $$

where $\Phi(X \le x\mid \mu,\sigma)$ is the Gaussian cdf.

Using this asymptotic distribution, the p-value for your hypothesis test is simply $1-P(\hat \Theta \le \hat \theta | \theta=1)$.

It should be noted that for any significance level less than $0.5$ (why would it not be?), using the above censored normal is equivalent to simply using the Gaussian pdf $\phi(\hat \theta \mid \mu=1,\sigma=1/\sqrt{n})$ which, as it happens, is the limiting distribution of $\bar X$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.