# What is the intuition behind exchangeable samples under the null hypothesis?

Permutation tests (also called a randomization test, re-randomization test, or an exact test) are very useful and come in handy when the assumption of normal distribution required by for instance, t-test is not met and when transformation of the values by ranking of the non-parametric test like Mann-Whitney-U-test would lead to more information being lost. However, one and only one assumption should not be overlooked when using this kind of test is the assumption of exchangeability of the samples under the null hypothesis. It is also noteworthy that this kind of approach can also be applied when there are more than two samples like what implemented in coin R package.

Can you please use some figurative language or conceptual intuition in plain English to illustrate this assumption? This would be very useful to clarify this overlooked issue among non-statisticians like me.

Note:
It would be very helpful to mention a case where applying a permutation test doesn't hold or invalid under the same assumption.

Update:
Supppose that I have 50 subjects collected from the local clinic in my district at random. They were randomly assigned to received drug or a placebo at 1:1 ratio. They were all measured for paramerter 1 Par1 at V1 (baseline), V2 (3 months later), and V3 (1 year later). All 50 subjects can be subgrouped into 2 groups based on feature A; A positive = 20 and A negative = 30. They can also be subgrouped into another 2 groups based on feature B; B positive = 15 and B negative = 35.
Now, I have values of Par1 from all subjects at all visits. Under the assumption of exchangeability, can I do comparison between levels of Par1 using permutation test if I would:
- Compare subjects with drug with those received placebo at V2?
- Compare subjects with feature A with those having feature B at V2?
- Compare subjects having feature A at V2 with those having feature A but at V3?
- By which situation this comparison would be invalid and would violate the assumption of exchangeability?

• Suppose I had each observation on a separate sheet of loose-leaf paper and as I handed you the stack, I slipped, and the sheets came flying out in all directions as they settled to the floor. It would be a shame if that destroyed the validity of the test you were hoping to perform on those data. If your observations are exchangeable and you were applying a test based on that, you'd comfort me and tell me not to worry while helping me collect up the papers off the floor. If not, and the data collection was especially expensive, I might need to run for my life. – cardinal Nov 13 '13 at 13:00
• On the other hand, order does matter for things like time-series data (in general) and tests should generally respect this order in an appropriate way. – cardinal Nov 13 '13 at 13:01
• @cardinal, while your intuitive story has drawn a vivid picture of how this assumption looks like, but I am still confused as how to judge whether the fallen valuable papers were exchangeable or not. You may run for another comment if it is possible! – doctorate Nov 13 '13 at 13:13

First, the non-figurative description: Exchangability means that the joint distribution is invariant to permutations of the values of each variable in the joint distribution (i.e, $f_{XYZ}(x = 1, y=3, z=2)=f_{XYZ}(x=3,y=2,z=1)$, etc). If this is not the case then counting permutations is not a valid way of testing the null hypothesis, as each permutation will have a different weight (probability/density). Permutation tests depend on each assignment of a given set of numerical values to your variables having the same density/probability.
A concrete example where exchangability is absent: You have N jars,each filled with 100 numbered tickets. The first M jars have tickets with only odd numbers from 1-200 (1 ticket per number), the remaining N-M have tickets for only even numbers between 1 - 200. If you select a ticket from each jar at random, you get a joint distribution on sample results. In this case, $f(x_1=1,x_2=2,X_3=3...X_N=N)\neq f(x_1=N,x_2=N-1,X_3=N-2...X_N=1)$ so you cannot just count permutations of the values 1 through N. In general, exchangability fails when your sample can be stratfied into sub-groups (as I have done with the jars). Exchangabilty would be restored if, insteas of taking 1 sample from N jars, you took N samples from 1 jar. Then, the joint distribuiotn would be invariant to permutations.