Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I want to find out if there is a calculation that can give me the minimum sample size to achieve a certain statistical power for a Permutation Test. I'm not assuming any parametric distribution for populations from where the samples come from.
I've been searching but haven't been able to find a satisfying answer. Statistical power seems like an overlooked aspect on most texts on permutation tests.
You partially answered your own question. Consider the reason you're often performing a permutation test. It's usually in circumstances where you have little faith in any particular parametric distribution or for some other reason want a non-parametric solution. In that case, how does one estimate power? You could be doing the permutation test in situations where the populations have multimodal distributions, uniform distributions, any combination for your conditions that don't match, etc. All of those might have different power functions.
Keep in mind that the way power is estimated requires an assumption of some world where you know effect size and distribution of the effect. Since you have no notion of the latter you can't estimate power a priori. The best you could do is estimate power for some particular distribution you might think is close, subset of likely distributions, or taking your data as the population. But the latter leaves you know way to get to power other than post hoc.
