Which test statistics is better in the hypothesis test for variance?

Question

Most of the time, to do hypothesis tests is to follow the tedious processes given on statistics textbooks. But recently I find that there are, in general, more than several test statistics that can be used to test the same hypothesis. To give an example consider the following test.

$n$ samples of the random variable $X$ are given. Null Hypothesis $H_0:X\sim N(\mu,\sigma_0^2)$. Alternative Hypothesis $H_1:X\sim N(\mu,\sigma_1^2)$. $\mu$ and $\sigma_i$ are known. $\sigma_0<\sigma_1$.

If we follow the textbooks, then we should use the $\chi$-square distribution, and the test statistic is the variance of the sample(times $n$ divided by $\sigma_0$). But the following test is also appropriate.

Let $\bar{X}$ be the sample mean. If $|\bar{X}-\mu|<c$ then accept $H_0$. Otherwise accept $H_1$. $c$ is chosen according to the level of significance of the test, $\alpha$.

This is an appropriate test because the condition $|\bar{X}-\mu|<c$ is more likely to be satisfied under $H_0$ than under $H_1$. (Many other test statistics are also appropriate for the same reason.) The test, however, has one shortcoming: the probability of making a type II error, $\beta$, is independent of $n$, which means it doesn't decrease as sample size increases.

Therefore I wonder which test statistic is the best in this situation. There is some simple criteria of how good a test statistic is. In this case, for example, we can say that for $n=100$ and $\alpha=0.05$, the smaller the minimum value of $\sigma_1/\sigma_0$ required to make $\beta\leq 0.05$, the better the test statistic.

Also, how can we make sure that the test given on the textbook is the best test, and there are no better alternatives?

Since this question might be too broad, I would be glad if anyone could give a specific example(like the one above) to illustrate the general idea. Also, are there any books or articles that discuss this topic in detail?

Answer 1

Different tests are generally compared by looking at their power, which is the probablity of rejecting the null hypothesis under different parameter values when the alternative hypothesis is actually true. Since it is correct to reject the null hypothesis when it is false, it is good to have higher power, and so if one test has higher power over all

In your model you have $X_1,...,X_n \sim \text{IID N}(\mu, \sigma)$ and your hypotheses are: $$H_0: \sigma = \sigma_0 \quad \quad \quad \quad \quad H_A: \sigma = \sigma_1,$$ where $\sigma_0 < \sigma_1$ are known values, and the mean $\mu$ is known. The tests you have described, taken at significance level $\alpha$, have rejection regions as follows. (In the specified critical point values we use subscript $\alpha$ to refer to the right-tail area of the distribution.)

Test 1: Reject the null hypothesis if $\sum (x_i-\mu)^2 \geqslant \sigma_0^2 \cdot \chi_{\alpha,n}^2$.

Test 2: Reject the null hypothesis if $|\bar{x}-\mu| \geqslant \sigma_0 \cdot z_{\alpha/2} / \sqrt{n}$.

We can compare these tests by finding the power of each of them under the specified variance in the alternative hypothesis. Tests 1-2 can be shown to have respective power:

$$\text{Power}_1(\alpha) = \mathbb{P}(\text{Reject }H_0 \text{ in Test }1|H_A) = \int \limits_{\sigma_0^2 / \sigma_1^2}^\infty \text{F-Dist}(r|n,n) dr.$$

$$\text{Power}_2(\alpha) = \mathbb{P}(\text{Reject }H_0 \text{ in Test }1|H_A) = \int \limits_{\sigma_0^2 / \sigma_1^2}^\infty \text{F-Dist}(r|1,1) dr.$$

It is simple to calculate the power of each of these tests based on the known ratio $\sigma_0^2/\sigma_1^2$ and the value $n$. The first test is more powerful than the second.