# Permutation tests for sharp vs weak null hypothesis

Permutation tests provide an easy way to obtain well-calibrated tests for experimental (randomized) data for the sharp null of no treatment effect.

The idea: In context where either only the treatment outcome $y_i(D_i=1)$ or the control outcome $y_i(D_i=0)$ is observed, and we care about the average treatment effect: $\hat{\tau} = \sum_{i|D_i=1}y_i-\sum_{i|D_i=0}y_i$ we can obtain it's distribution under the null of no treatment effect $$y_i(1)=y_i(0),$$ simply be repeatedly permuting the treatment vector $D$ and computing $\hat{\tau}$ (or the corresponding t-statistic). This is often referred to as a test of the "sharp null hypotheses", as opposed to the "weak null hypothesis" which would be something like "no average treatment effect":$$E[y_i(1)]=E[y_i(0)]$$

My question: I came across papers [1, 2] which seem to imply that under certain assumptions asymptotically this is also a valid test of the "weak" hypothesis of no average treatment effect. I am now looking for a canonical/text-book reference for this statement, in particular w.r.t. the assumptions under which this is valid.

References:

[1] Chung, EunYi, and Joseph P. Romano. "Exact and asymptotically robust permutation tests." The Annals of Statistics 41.2 (2013): 484-507. APA (https://arxiv.org/pdf/1304.5939.pdf)

[2] Bugni, Federico A., Ivan A. Canay, and Azeem M. Shaikh. "Inference under covariate-adaptive randomization." Journal of the American Statistical Association just-accepted (2017).

EDIT: Additional, related reference that is sometimes cited in that context, unclear why: Janssen, Arnold. "Studentized permutation tests for non-iid hypotheses and the generalized Behrens-Fisher problem." Statistics & probability letters 36.1 (1997): 9-21. (http://www.sciencedirect.com/science/article/pii/S0167715297000436)

• Consider a mixture of treatment effects, where the mixture components are symmetric about 0 under the null but biased toward larger values (or toward smaller values) under the alternative. That seems to be a case where it would work in that way. – Glen_b Nov 28 '17 at 8:49