# A stopping rule for permutation test

I perform a permutation test multiple times on different datasets, each time I am only concerned about significant $p$ values. To reduce computation time would it be correct to introduce this kind of stopping rule: After a certain number of $N$ permutations to check whether $p$ is greater than a particular value. So, for example if $p>0.1$ after $N$=200 permutations then the lower bound of 95% confidence interval would be greater than $0.05$. Therefore, calculations could be stopped as a true $p$ is not significant. Just want to make sure I am doing it right. Thank you.

• are your permutations properly randomised, if that makes sense in your context? – Robert Jones Jun 7 '13 at 11:47
• yes, all permutations are independent. Each time I do permutation of initial data set, I calculate the new value of statistics, and count the number of cases when statistics from permutation is greater or equal than initial statistics. So, $p$ is a proportion of such cases. – Math_cat Jun 7 '13 at 12:08

• The idea was to stop simulation if the 95% confidence interval of estimated $p$ value is located above 0.05 after particular number of permutations. I am using the normal approximation to obtain the CI in this case. So, no, I don't keep simulating in this case. Thank you for pointing out the SPRT approach. Initially, I performed around 2500 permutations in order to estimate p-value of order of 0.01. – Math_cat Jun 7 '13 at 12:05