In computer science, mathematics, and sometimes in other fields, “esoteric” examples cannot only be entertaining, but helpful to illustrate certain concepts, for example:
Esoteric programming languages demonstrate how far-reaching the concept of a programming language is and help to appreciate good programming languages.
I am currently preparing some teaching on using hypothesis tests and think that having a test with very low power (but no other flaws) would help to illustrate the concept of statistical power. (Of course, I still have to decide myself whether a given example is didactically useful for my audience or just confusing.)
Are there any statistical test with intentionally low power, more specifically:
- The test fits in the general framework of hypothesis tests, i.e., it works with a null hypothesis, has requirements, and returns a (correct) p value.
- It is not intended/proposed for serious application.
- It has an very low power (due to an intentional design flaw and not due to low sample or effect size).
If you can fundamentally argue that such a test cannot exist, I would also consider this a valid answer to my question. If on the other hand, a plethora of such tests exists, I am interested in the most didactically efficient one, i.e., it should be easily accessible and have a striking effect.
Note that I am not asking for a general selection of statistical mistakes (cherry picking, etc.) or similar.
What I found so far
Internet searches returned nothing for me.
Every attempt to construct something like this ended either up in some (useful) existing test or the format is not that of a regular test. For example, I thought about a test whether a population has a positive median that returns only yes if all samples are positive; but that test does not return a p value and thus does not fit within the usual test framework. If I just count the positive and negative signs as a test statistic (and compute the p values accordingly), I end up with the sign test, which is a reasonable test.