3
$\begingroup$

Permutation tests provide an easy way to obtain well-calibrated tests for experimental (randomized) data for the sharp null of no treatment effect.

The idea: In context where either only the treatment outcome $y_i(D_i=1)$ or the control outcome $y_i(D_i=0)$ is observed, and we care about the average treatment effect: $\hat{\tau} = \sum_{i|D_i=1}y_i-\sum_{i|D_i=0}y_i$ we can obtain it's distribution under the null of no treatment effect $$y_i(1)=y_i(0),$$ simply be repeatedly permuting the treatment vector $D$ and computing $\hat{\tau}$ (or the corresponding t-statistic). This is often referred to as a test of the "sharp null hypotheses", as opposed to the "weak null hypothesis" which would be something like "no average treatment effect":$$E[y_i(1)]=E[y_i(0)]$$

My question: I came across papers [1, 2] which seem to imply that under certain assumptions asymptotically this is also a valid test of the "weak" hypothesis of no average treatment effect. I am now looking for a canonical/text-book reference for this statement, in particular w.r.t. the assumptions under which this is valid.

References:

[1] Chung, EunYi, and Joseph P. Romano. "Exact and asymptotically robust permutation tests." The Annals of Statistics 41.2 (2013): 484-507. APA (https://arxiv.org/pdf/1304.5939.pdf)

[2] Bugni, Federico A., Ivan A. Canay, and Azeem M. Shaikh. "Inference under covariate-adaptive randomization." Journal of the American Statistical Association just-accepted (2017).

EDIT: Additional, related reference that is sometimes cited in that context, unclear why: Janssen, Arnold. "Studentized permutation tests for non-iid hypotheses and the generalized Behrens-Fisher problem." Statistics & probability letters 36.1 (1997): 9-21. (http://www.sciencedirect.com/science/article/pii/S0167715297000436)

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Consider a mixture of treatment effects, where the mixture components are symmetric about 0 under the null but biased toward larger values (or toward smaller values) under the alternative. That seems to be a case where it would work in that way. $\endgroup$
    – Glen_b
    Commented Nov 28, 2017 at 8:49

1 Answer 1

Reset to default
2
$\begingroup$

The key is to use the studentized test statistic in a randomization test. Here are two papers under the finite-population framework with the treatment assignment being the only random component.

Ding, P. and Dasgupta, T. (2018). A Randomization-Based Perspective on Analysis of Variance: A Test Statistic Robust with Respect to Treatment Effect Heterogeneity. Biometrika, 105, 45-56.

Wu, J. and Ding, P. (2018). Randomization tests for weak null hypotheses. ArXiv: https://arxiv.org/abs/1809.07419

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks, you should probably point out that these are your papers. But your answer is much appreciated $\endgroup$
    – sheß
    Commented Jan 9, 2019 at 8:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.