I see optimality mentioned in the context of both Neyman-Pearson tests and Wald tests. Neyman Pearson is a fixed sample test, while Wald is a sequential test. The Neyman-Pearson lemma states that no test which has a smaller Type I error can also have a smaller Type II error than the LRT. Wald states that no test which has a smaller Type I and Type II error can have a smaller expected stopping time. Are these two different criteria of optimality, and hence can co-exist?

The standard power analysis taught in statistics textbooks requires as input $\alpha$ (Type I error), $\beta$ (Type II error), and $\Delta$ (difference of means), and returns a fixed sample size $N$. Wald on the other hand only requires $\alpha$ and $\beta$. Does Wald's optimality theorem mean that it will require smaller number of samples on average than those required by the power analysis procedure? This would be very surprising because Wald is $\Delta$-agnostic.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.