In practice, how do you find the uniformly most powerful test? Would you essentially brute-force all possible hypothesis tests?

Could we prove that there exists a uniformly most powerful test and the one that we are using is sub-optimal?


For a case of testing simple hypothese, there's a Neyman–Pearson lemma whereas for case of composite hypotheses we've got Karlin–Rubin theorem, which is a bit limiting (to scalar parameters and scalar measurements). Probably there're other more general solution instead of Karlin–Rubin theorem, but unfortunately they're unknown to me. I recommend you to have a look on book by E.L. Lehmann and J.P. Romano Testing Statistical Hypotheses (the whole 3rd chapter is about UMP, at least in 3rd edition).

Let's have a glance on the Neyman-Person lemma and go to a working example that should give you a vague insight how to create such UMP test in some cases:

We're considering a random variable $X \sim \mathcal{N}(0, \sigma^2)$ and an one-sided test: $$H_0: \sigma^2 \leq \sigma_0^2 ~~~ \text{against} ~~~ H_A: \sigma^2 > \sigma_0^2 $$ We have a sample of $n$ independent random variables of common distribution $$X_1, \dots, X_n \sim \mathcal{N}(0, \sigma^2)$$

Now, we're computing a likelihood ratio for this sample: $$ \frac{L(\sigma_2^2|X_1, \dots, X_n)}{L(\sigma_1^2|X_1, \dots, X_n)} = \frac{\frac{1}{(\sqrt{2 \pi\sigma_2^2})^n} \cdot \exp(-\frac{1}{2\sigma_2^2} \sum\limits_{k=1}^n X_k^2)}{\frac{1}{(\sqrt{2 \pi\sigma_1^2})^n} \cdot \exp(-\frac{1}{2\sigma_1^2} \sum\limits_{k=1}^n X_k^2)} $$

$$ \frac{L(\sigma_2^2|X_1, \dots, X_n)}{L(\sigma_1^2|X_1, \dots, X_n)} = (\frac{\sigma_1}{\sigma_2})^n \cdot \exp[(\frac{1}{2\sigma_1^2} - \frac{1}{2\sigma_2^2}) \cdot \sum\limits_{k=1}^n X_k^2] $$ In this form, we clearly see that the likelihood ratio is monotonically increasing with respect only to the statistic $$ T = \sum\limits_{k=1}^n X_k^2 $$

Using Neyman-Person lemma (look at this particular form Theorem 1: Neyman-Person Lemma) it can be said that there's a critical region of the form $$ C = \{ (X_1, \dots, X_n) | \sum\limits_{k=1}^n X_k^2 \geq CritVal_{\alpha}\} $$ for a Uniformly Most Powerful Test with significance level of $\alpha$.

Now, we must only find a critical value for given $\alpha$ level. It's easy to see that $$ \frac{1}{\sigma_0^2} \sum\limits_{k=1}^n X_k^2 \sim \chi_n^2 ~~~~ (\chi^2~\text{with $n$ degrees of freedom}) $$

We can introduce an auxiliary random variable $A \sim \chi_n^2$ and find such value of $t$ satisfying $$ P(A > t) = \alpha $$ then we can state $CritVal_{\alpha} = t \cdot \sigma_0^2$.

To sum up, we've just constructed a critical region, so we have actual UMP test for this particular task.

  • $\begingroup$ Wow, nice! So this will confirm whether or not we have a a UMP test. Now we must just try them all out right and determine whether they are equivalent? $\endgroup$ – user46925 Feb 29 '16 at 0:35
  • $\begingroup$ @zero I'm not sure what are you trying to achieve, so maybe show us what kind of tests you're coping with. $\endgroup$ – Adam Przedniczek Feb 29 '16 at 0:52
  • $\begingroup$ I have no tests in consideration. I want to determine which test will provide the best power. $\endgroup$ – user46925 Feb 29 '16 at 0:56
  • 1
    $\begingroup$ @zero No test = no problem ;-) , but seriously, in many situations it would be NO UMP test among all tests of $\alpha$ level. In such situation, you can narrow down the set of considered tests to UMP Unbiased tests. See chapter 4th of the book I mentioned above. However, if you have or not an UMP or UMP Unbiased test, if I were you, I would conduct a small Monte Carlo simulation to find out a real required sample size for a particular power and significance level. It's always better to know in advance the minimum number of samples you must gather to conduct a hypoteses testing you can rely on. $\endgroup$ – Adam Przedniczek Feb 29 '16 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy