How to define a Rejection Region when there's no UMP?

Question

Consider the linear regression model

$\mathbf{y}=\mathbf{X\beta}+\mathbf{u}$,

$\mathbf{u}\sim N(\mathbf{0},\sigma^2\mathbf{I})$,

$E(\mathbf{u}\mid\mathbf{X})=\mathbf{0}$.

Let $H_0: \sigma_0^2=\sigma^2$ vs $H_1: \sigma_0^2\neq\sigma^2$.

We can deduce that $\frac{\mathbf{y}^T\mathbf{M_X}\mathbf{y}}{\sigma^2}\sim \chi^2(n-k)$, where $dim(\mathbf{X})=n\times k$. And $\mathbf{M_X}$ is the typical notation for the annihilator matrix, $\mathbf{M_X}\mathbf{y}=\hat{\mathbf{y}}$ , where $ \hat{\mathbf{y}}$ is the dependent variable $\mathbf{y}$ regressed on $\mathbf{X}$.

The book I'm reading states the following:

I've previously asked what criteria should be used to define a Rejection Region(RR), see the answers to this question, and the main one was to choose the RR that made the test as powerful as possible.

In this case, with the alternative being a bilateral composite hypothesis there's usually no UMP test. Also, by the answer given in the book, the authors don't show if they did a study of the power of their RR. Nevertheless, they chose a two-tailed RR. Why is that, since the hypothesis do not 'unilaterally' determine the RR?

Edit: This image is in the solution manual of this book as the solution to exercise 4.14.

Answer 1

Easier to first work through the case where the regression coefficients are known & the null hypothesis therefore simple. Then the sufficient statistic is $T=\sum z^2$, where $z$ is the residual; its distribution under the null is also a chi-squared scaled by $\sigma^2_0$ & with degrees of freedom equal to the sample size $n$.

Write down the ratio of the likelihoods under $\sigma=\sigma_1$ & $\sigma=\sigma_2$ & confirm that it's an increasing function of $T$ for any $\sigma_2 > \sigma_1$:

The log likelihood ratio function is $$\ell(\sigma_2;T,n)-\ell(\sigma_1;T,n)=\frac{n}{2} \cdot \left[\log \left(\frac{\sigma_1^2}{\sigma_2^2}\right) + \frac{T}{n} \cdot \left(\frac{1}{\sigma_1^2} - \frac{1}{\sigma_2^2}\right) \right]$$, & directly proportional to $T$ with positive gradient when $\sigma_2>\sigma_1$.

So by the Karlin–Rubin theorem each of the one-tailed tests $H_0:\sigma=\sigma_0$ vs $H_\mathrm{A}:\sigma < \sigma_0$ & $H_0:\sigma = \sigma_0$ vs $H_\mathrm{A}:\sigma < \sigma_0$ is uniformly most powerful. Clearly there's no UMP test of $H_0:\sigma = \sigma_0$ vs $H_\mathrm{A}:\sigma \neq \sigma_0$. As discussed here, carrying out both one-tailed tests & applying a multiple-comparisons correction leads to the commonly used test with equally sized rejection regions in both tails, & it's quite reasonable when you're going to claim either that $\sigma>\sigma_0$ or that $\sigma<\sigma_0$ when you reject the null.

Next find the ratio of the likelihoods under $\sigma=\hat\sigma$, the maximum-likelihood estimate of $\sigma$, & $\sigma=\sigma_0$:

As $\hat\sigma^2=\frac{T}{n}$, the log likelihood ratio test statistic is $$\ell(\hat\sigma;T,n)-\ell(\sigma_0;T,n)=\frac{n}{2} \cdot \left[\log \left(\frac{n\sigma_0^2}{T}\right) + \frac{T}{n\sigma_0^2} - 1 \right]$$

This is a fine statistic for quantifying how much the data support $H_\mathrm{A}:\sigma \neq \sigma_0$ over $H_0:\sigma = \sigma_0$. And confidence intervals formed from inverting the likelihood-ratio test have the appealing property that all parameter values inside the interval have higher likelihood than those outside. The asymptotic distribution of twice the log-likelihood ratio is well known, but for an exact test, you needn't try to work out its distribution—just use the tail probabilities of the corresponding values of $T$ in each tail.

If you can't have a uniformly most powerful test, you might want one that's most powerful against the alternatives closest to the null. Find the derivative of the log-likelihood function with respect to $\sigma$—the score function:

$$\frac{\mathrm{d}\,\ell(\sigma;T,n)}{\mathrm{d}\,\sigma}=\frac{T}{\sigma^3} - \frac{n}{\sigma}$$

Evaluating its magnitude at $\sigma_0$ gives a locally most powerful test of $H_0:\sigma=\sigma_0$ vs $H_\mathrm{A}:\sigma \neq \sigma_0$. Because the test statistic's bounded below, with small samples the rejection region may be confined to the upper tail. Again, the asymptotic distribution of the squared score is well known, but you can get an exact test in the same way as for the LRT.

Another approach is to restrict your attention to unbiased tests, viz those for which the power under any alternative exceeds the size. Check your sufficient statistic has a distribution in the exponential family; then for a size $\alpha$ test, $\phi(T)= 1$ if $T<c_1$ or $T>c_2$, else $\phi(T)= 0$, you can find the uniformly most powerful unbiased test by solving $$\begin{align} \operatorname{E}(\phi(T)) &= \alpha \\ \operatorname{E}(T\phi(T)) &= \alpha \operatorname{E} T \end{align} $$

A plot helps show the bias in the equal-tail-areas test & how it arises:

At values of $\sigma$ a little over $\sigma_0$ the increased probability of the test statistics' falling in the the upper-tail rejection rejection doesn't compensate for the reduced probability of its falling in the lower-tail rejection region & the power of the test drops below its size.

Being unbiased is good; but it's not self-evident that having a power slightly lower than the size over a small region of the parameter space within the alternative is so bad as to rule out a test altogether.

Two of the above two-tailed tests coincide (for this case, not in general):

The LRT is UMP among unbiased tests. In cases where this isn't true the LRT may still be asymptotically unbiased.

I think all, even the one-tailed tests, are admissible, i.e. there's no test more powerful or as powerful under all alternatives—you can make the test more powerful against alternatives in one direction only by making it less powerful against alternatives in the other direction. As the sample size increases, the chi-squared distribution becomes more & more symmetric, & all the two-tailed tests will end up being much the same (another reason for using the easy equal-tailed test).

With the composite null hypothesis, the arguments become a little more complicated, but I think you can get practically the same results, mutatis mutandis. Note that one but not the other of the one-tailed tests is UMP!

Answer 2

In this case, with the alternative being a bilateral composite hypothesis there's usually no UMP test.

I am not sure if that is true in general. Certainly, a lot of the classical results (Neymon-Pearson, Karlin-Rubin) are based on either simple or one-sided hypothesis, but generalizations to two-sided composite hypothesis do exist. You can find some notes on that here, and more discussion in the textbook here.

For your problem specifically, I don't know whether a UMP test exists or not. But intuitively, it seems to be that under 0-1 loss, a one sided test will probably be inadmissible, and thus the class of admissible test will be all two-sided tests. Give the class of two sided tests, the goal is to find the one with the largest power, which should automatically happen by choosing quantiles around the one mode of the $\chi^2$. (This is all based on intuition).