# Is exact power analysis of the permutation (or randomization) test possible without i.i.d assumptions on the data?

In hypothesis testing, we have the null hypothesis ($H_0$) and the alternate hypothesis ($H_1$). The null hypothesis typically states that units drawn from the two groups have identical outcomes, whereas the alternate states that they differ. The Permutation test rejects the null $H_0$ if the computed p-value is less than the significance criterion (say $\alpha$). The permutation test itself does not require assumptions of i.i.d, rather a weaker assumption of exchangeability under the null hypothesis is enough for its application.

Power is P[rejection | $H_1$]. Is it possible to compute power without making i.i.d assumptions? I understand that distributional assumptions (like normality) will make it easier to compute the power, but I'm trying to understand the minimum set of assumptions needed. For example, if one assumes independence and the data is binary, a particular form of $H_1$ makes the outcomes follow the Binomial distribution. For example, $H_1: p_1=0.4, p_2=0.6$, where $p_i$ is the probability of a unit in the $i^{th}$ group getting outcome $1$. Thus, with just the independence assumption, power can be computed for the specific form of $H_1$.

I can see the assumption of ‘identically distributed’ may be necessary. If the alternative hypothesis is indeed false, we cannot draw any conclusions about the distribution of measurements in the sample unless the measurements are identically distributed. One example of non-identically distributed outcomes is that outcomes are drawn such that all units get $1$ - leading to a certain failure in rejecting the null. Unless $H_1$ is so extreme that it makes it impossible for one group to get the outcome $1$, this drawing is possible. With identically distributed measurements, this particular drawing is still possible, but it has a low probability (if the alternate is true), thereby leading to a rejection of the null.

But, is independence necessary?

Edit: Monte Carlo simulations are a way to estimate power, but that requires model assumptions on how real world data is generated. I reworded the question to indicate that we are interested in an exact power computation.