2
$\begingroup$

I've found that the asymptotic LR test is used in simple vs bilateral hypothesis test in which it is impossible to actually compute the rejection region, or better, in which we would need to find a numerical solution for the two cutoffs.

For example, we have a sample $X_1,...,X_n\sim N(\mu,\sigma^2)$, both unknown. We want to test $H_0:\sigma=\sigma_0, H_1:\sigma\neq \sigma_0$. Computing the test statistic, I obtained a rejection rule as $\frac{\hat \sigma^2}{\sigma_0^2}exp\{1-\frac{\hat \sigma^2}{\sigma_0^2}\} \leq k_\alpha$, where $\hat \sigma$ is the MLE for $\sigma$.

The rejection region has the form of $g(u)=ue^{(1-u)}, u>0$. Now, I know that we reject the null hypothesis for values of $u>k'_\alpha$ and $u<k''_\alpha$ but I don't know how to compute $k'_\alpha$ and $k''_\alpha$.

How should I proceed using the asymptotic LR test? Is that a case in which it can be applied? Are there any other cases?

$\endgroup$
  • $\begingroup$ How does your test statistic (a function of $u$) relate to your sample? $\endgroup$ – Glen_b Jan 4 '16 at 10:22
7
$\begingroup$

I don't think your derivation of the likelihood ratio test is correct. Let's start from the beginning. I will write everything in terms of the variance since this way we can use some known results about normal distributions. This does not change the nature of the problem either.

We wish to test

$$ H _0 : \sigma^2 = \sigma_0^2 \quad \text{vs} \quad \sigma^2 \neq \sigma_0^2 $$

for a normal model, i.e. $X\sim N\left( \mu, \sigma^2 \right)$. Let $\omega$ denote the restricted set of parameters and $\Omega$ the unrestricted one. It is then easy to see that under $H_0$,

$$\omega = \left\{ -\infty<\mu <\infty, \ \sigma^2 = \sigma^2_0 \right\}$$

while

$$\Omega = \left\{ -\infty<\mu <\infty, \sigma^2 > 0 \right\} $$

Note that the second set is much larger since we are also allowed to vary $\sigma^2$. In fact, we can maximize in two dimensions under $\Omega$. Assuming then we have a sample of iid observations we will be looking at the quotient

$$ L = \frac{ \sup_{\mu, \sigma^2 \in \omega} f \left( \mathbf{x}, \mu, \sigma^2 \right) }{\sup_{\mu, \sigma^2 \in \Omega} f \left( \mathbf{x}, \mu, \sigma^2 \right)} $$

and we will be rejecting the null hypothesis for low values. In most cases, only an asymptotic rejection rule may be obtained from this but since we are dealing with a normal distribution, the problem becomes quite tractable. So let's maximize and see what we got.

In $\omega$ you can verify that the optimization yields $\hat{\mu} = \bar{x}$ and $\hat{\sigma}^2 = \sigma^2_0$, no room to maneuver here, while in $\Omega$ we get the regular mle solution, namely $\hat{\mu} = \bar{x}$ and $\hat{\sigma}^2 = n^{-1} \sum_{i=1}^n \left(x_i - \bar{x} \right)^2$. Insert these values into the likelihood ratio to obtain the rejection rule

$$ \frac{ \left( \frac{1}{\sigma_0^2} \right) ^{n/2} \exp \left\{ - \frac{1}{2\sigma^2_0} \sum_{i=1}^n \left(x_i - \bar{x} \right)^2 \right\}} {\left( \frac{1}{ \hat {\sigma}^2} \right) ^{n/2} \exp\left\{-\frac{n}{2} \right\} } \leq c \tag{1}$$

which after merging constants and simplifying is equivalent to

$$ \left( \frac{\hat{\sigma}^2}{\sigma_0^2} \right)^{n/2} \exp\left\{ - \frac{n}{2} \frac{\hat{\sigma}^2}{\sigma_0^2} \right\} \leq k $$

Thus we are left with a function $f(x) = x^c e^{-cx}$ which is unimodal with maximum at $x=1$. Here is what it looks like

enter image description here

From this, we may conclude that the null hypothesis will be rejected for too small or too large values of $x$. But $x$ is just $\frac{\hat{\sigma}^2}{\sigma_0^2}$, hence the corresponding rejection regions are

$$\frac{\hat{\sigma}^2}{\sigma_0^2} \leq k_1 \quad \text{or} \quad \frac{\hat{\sigma}^2}{\sigma_0^2} \geq k_2 $$.

All that remains now is to determine the distribution of this quantity, under $H_0$, i.e. for $\sigma_0^2 = \sigma^2$. Recalling that for normal models

$$\frac{ \left(n-1\right) S^2}{\sigma^2} \sim \chi^2 (n-1)$$

we can find quantiles from the $\chi^2$ distribution such that we have a 5% level test, which would then call us to reject $H_0$ if

$$ \frac{ \left(n-1\right) S^2}{\sigma_0 ^2} < \chi^2 _{0.025} (n-1) \quad \text{or} \quad \frac{ \left(n-1\right) S^2}{\sigma_0 ^2} > \chi^2 _{0.975} (n-1) $$

By the way, the result is exact so you don't need the asymptotics. It is often the case that such results hold exactly for the normal distribution but you would need the asymptotic $\chi^2$ otherwise.

Hope this helps.

$\endgroup$
  • $\begingroup$ I don't get how could you come with a function $f(x)=x^ce^{-cx}$? If you go on with the computations, I am pretty sure you would obtain something in the form of $f(x)=xe^{(1-x)}$. Then, I am ok with the rejection rule you find, that is the same I obtained. $\endgroup$ – PhDing Jan 4 '16 at 18:22
  • $\begingroup$ @Alessandro Can you get equation (1)? If you then simplify you will get this function. $\endgroup$ – JohnK Jan 4 '16 at 18:29
  • $\begingroup$ I get that but I didn't get the further simplifications. But maybe I don't get the point: have you put the $c$ in the rhs of the inequality together with lhs? $\endgroup$ – PhDing Jan 4 '16 at 18:37
  • $\begingroup$ @Alessandro No, that's unrelated. I filled in the missing line. $\endgroup$ – JohnK Jan 4 '16 at 18:43
  • 1
    $\begingroup$ @Alessandro This is a quite advanced topic that is treated in Lehmann's book. From what I know, the LRT is not optimal but it comes very close to being so. $\endgroup$ – JohnK Jan 5 '16 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.