Wikipedia states (without citation):

Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests.

What are examples of exact tests which are not permutation tests? I assume that Fisher's exact test is also considered to be a permutation test since it looks to my understanding at all possible permutations of contingency tables (see also here). Is this correct?


Any test that keeps the Type I error rate of the test at the specified level is exact. In this sense, for example, the well known one-sample t test for normally distributed data is exact. Similarly, the two-sample t test for normally distributed data of equal variance is exact.

Actually, most of the "standard" tests are exact on their more restrictive formulations.

  • $\begingroup$ I'm not sure that this is correct. I thought an 'exact' test is one where the assumptions do not include any approximations, such as an assumption that the data or errors are normally distributed. That is quite different from simply providing specified alpha under an assumption that can only be approximately true. $\endgroup$ Dec 20 '19 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.