Say using the sampling distribution of the same mean, we standardize and obtain the standard normal $N(0,1)$ value: $$ Z=\frac{\bar{X}-\mu}{\sqrt{\frac{\sigma}{n}^{2}}} $$ We wish to test the hypothesis $H_{0}\mu=a;$ $H_{1}:$$\mu>a.$ After computing the statistic, we find the critical value corresponding to a specific chosen significance level $\alpha$, $Z_{\alpha},$ such that if $Z>Z_{\alpha}$, we reject. One potential issue here is that if we reduce $\alpha,$ we would increase $\beta$ (Type II error). My question is this: $Z_{\alpha}$ here is computed as the smallest value such $Z_{\alpha}$ such that $$ \mathbb{P}\left(Z>Z_{\alpha}\right)=0.05 $$ In other words, we choose $Z_{\alpha}$ such that: $$ \int_{Z_{\alpha}}^{\infty}f(Z,\mu_{0},\sigma_{0}^{2})dZ=0.05 $$ where $f\left(.\right)$ is the standard normal pdf, and $\mu_{0}$ and $\sigma_{0}^{2}$ are the distribution of the mean and variance parameters under the null. Imagine an alternative definition, $$ \text{New Critical Region=}\int_{a}^{x_{1}}f(Z,\mu,\sigma^{2})dZ+\int_{x_{2}}^{x_{3}}f(Z,\mu,\sigma^{2})dZ+...+\int_{x_{n-1}}^{x_{n}}f(Z,\mu,\sigma^{2})dZ=0.05 $$ where the $x_{1},x_{2},x_{3}$ each represent points on the real line, greater than $\alpha,$ such that this new critical region can be envisioned as the sum of separate ``pockets'' of critical regions. The advantage, it seems to me, would perhaps be that we could increase our power (decrease probability of Type II error), while maintaining current significance levels. In other words, we could perhaps define a constrained mimization problem, such that, for a given alternative (say we have some knowledge a priori), we define $$ \text{min}_{x_{1},...x_{n}}\text{Type II error } $$ such that $$ \int_{a}^{x_{1}}f(Z,\mu,\sigma^{2})dZ+\int_{x_{2}}^{x_{3}}f(Z,\mu,\sigma^{2})dZ+...+\int_{x_{n-1}}^{x_{n}}f(Z,\mu,\sigma^{2})dZ=0.05 $$

This seems “mechanical” and counter-intuitive, but I was wondering if similar tests have been proposed.

  • $\begingroup$ Having $\alpha$ as the lower bound of the first integrall looks wrong. $\alpha$ is a probability, but the $x_i$ live in the observation space. You may mean $Z_\alpha$, but in fact in order to respect the level $\alpha$ under general choice of the other $x_i$, it should be a general value $x_0$. Also it's confusing that you have $x_{n-1}$ as upper bound of the last integral. $\endgroup$ – Lewian May 4 at 21:22
  • $\begingroup$ Sorry- it should be an a, not \alpha. $\endgroup$ – ChinG May 4 at 21:37
  • 1
    $\begingroup$ A somewhat related note: A small region centered at 0 with null probability $\alpha$ is sometimes used to flag results that are "too good to be true," eg, academic misconduct. $\endgroup$ – BigBendRegion May 4 at 23:47

The standard Gauss-test with the standard rejection region is a uniformly most powerful test, as was shown originally by Neyman and Pearson, see for example Lehmann and Romano's "Testing Statistical Hypotheses". So what you propose here will not increase the power, but in fact will make it worse, unless you choose your $x_1,\ldots,x_n$ in such a way that it's equivalent to the standard test, i.e. all your intervals together are just $[Z_\alpha,\infty]$ . For this reason such tests are not considered.

In other words, solving your minimisation problem will just reproduce the standard test (as long as you allow $x_n=\infty$).

Some words on why that holds: The likelihood ratio of the distribution of the test statistic between $\mu=a$ and $\mu=b$ for any $b>a$ is monotonic, meaning that generally the higher the value of the test statistic, the stronger evidence it provides against $\mu=a$. If there were a "hole" in the rejection region, this would mean to not reject $\mu=a$ at a certain larger value of the test statistic when it is rejected at a smaller value. This is inefficient trying to make the type II error probability as small as possible, because according to the likelihood (directly related to the type II error probability, which integrates the likelihoods) that larger value is more likely under the alternative compared to the null (quotient of likelihoods) than the smaller value.

  • $\begingroup$ Thank you! I will check it out. $\endgroup$ – ChinG May 4 at 21:39
  • $\begingroup$ +1. To add some intuition (hopefully), the standard way of designing rejection regions amounts to rejecting when the statistic is "extreme" relative to the null distribution, a reasonable property that would get lost under your proposal. $\endgroup$ – Christoph Hanck May 5 at 14:48
  • $\begingroup$ @ChristophHanck I agree with that. Perhaps it is this intuition of the rejection region based on extremities that leads to the NP UMP test result. $\endgroup$ – ChinG May 5 at 19:12
  • 2
    $\begingroup$ I have added an explanation that makes more direct reference to how this is proved. I'd say that your intuition is correct, if in some distance from how the proof works. $\endgroup$ – Lewian May 5 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.