4
$\begingroup$

In the book "Common errors in statistics" https://www.amazon.com/Common-Errors-Statistics-Avoid-Them/dp/1118294394, I read the following statement

Permutation tests only yield exact significance levels if the labels on the observations are weakly exchangeable under the null hypothesis. Thus, they cannot be successfully applied to the coefficients in a multivariate regression.

I don't think I understand what is meant by "weakly exchangeable observations". What would be an example where they are weakly exchangeable? What would be a more concrete counterexample than the coefficients in a multivariate regression? What kind of (invalid) permutation test could be applied to the coefficients in a multivariate regression at all?


As a subquestion: In my current analysis, I am doing a metastudy and I am considering a permutation test. Each observation is a result published in a separate primary study. Can I assume that my observations are weakly exchangeable? (I think yes, because each result can have been found in any other primary study).

$\endgroup$
5
  • 1
    $\begingroup$ Have you seen a definition of (weak) exchangeability? $\endgroup$
    – Glen_b
    May 6, 2016 at 13:44
  • 1
    $\begingroup$ No, I tried to search for it, but got mostly papers on something called Hoeffding-ANOVA decompositions (which apply the concept, don't define it) and a google book result for the exact book I cited. $\endgroup$
    – rumtscho
    May 6, 2016 at 13:48
  • 1
    $\begingroup$ Did you read the wikipedia article on exchangeability? $\endgroup$
    – Glen_b
    May 6, 2016 at 14:06
  • 1
    $\begingroup$ Beware, there are some issues here: 1. As it says in the quote the requirement of weak exchangeability only applies under the null. 2. the notion applies to the random variables from which the observations are drawn, not to the observed values. 3. You can't necessarily tell if it's satisfied - the actual random variables from which the observations were drawn might not obey that requirement (because if the null is false they won't be exchangeable) - it (like the stronger condition of i.i.d) is an assumption you might be able to give some argument for. $\endgroup$
    – Glen_b
    May 6, 2016 at 14:08
  • 1
    $\begingroup$ I discuss permutation testing (with a little handwaving) in the univariate regression case here; that might help for some context. Whether you could do it in the multivariate context would depend on what things you treated as the random variables you were trying to exchange, and what the null was. $\endgroup$
    – Glen_b
    May 6, 2016 at 14:22

1 Answer 1

2
$\begingroup$

I did not see this term before, but my guess is that it is used in opposition to the term "infinitely exchangeable". The latest is necessary for the deFinetti representation theorem. As you have seen this term in the context of permutation tests, thst seems reasonable, since permutation tests do not need the stronger infinite exchangeability, as it only uses probability calculations for the fixed sample size $n$, no asymptotic calculations which might need the stronger concept. See Can someone explain the concept of 'exchangeability'? for more discussion on exchangeability.

As for the question on permutation tests for regression parameters, have a look at the book A Chronicle of Permutation Statistical Methods 1920–2000, and Beyond by Kenneth J. Berry, Janis E. Johnston, and Paul W. Mielke Jr. Its section 5.32 Kennedy–Cade and Multiple Regression gives some permutation tests for multiple regression, see this paper.


More formally, we say that the random vector $X=(X_1,X_2, \dotsc, X_n)$ is exchangeable (or weakly exchangeable, if my guess is correct) if all permutations of the components of $X$ have the same distribution, that is, $$ (X_{\pi 1},X_{\pi 2}, \dotsc, X_{\pi n}) $$ have the same distribution as $X$ for all permutations $\pi$ of $\{1,2,\dotsc,n\}$. This condition is not enough for the validity of deFinetti's representation theorem. For that we need infinite exchangeability, meaning that for all $m>n$ there exist a random vector $Y$, say, of dimension $m-n$, such that the distribution of $(X,Y)$ is exchangeable in the above sense.

An example where this extension is not possible is an equicorrelated multinormal vector with, say, expectation vector 0 and covariance matrix with all diagonal elements 1 and all off-diagonal elements the value $\rho <0$. If this multinormal vector has dimension $n$, there is an inequality $$ \rho \ge -\frac1{n-1} $$ Say, $n=3$ and $\rho=-0.1$. The above inequality gives $\rho \ge -0.5$, so is fulfilled. But then we try to extend with the new vector $Y$, we have the inequality $$ -0.1 \ge -\frac1{m-1} $$ which solved give $m \le 11$. So we can extend $X$ with new components until length 11, but then it stops, so $X$ is not infinitely exchangeable. But if my guess is correct, we can say that $X$ is weakly exchangeable. And, in the context of permutation tests, that looks reasonable. (In the multaivariate normal equicorrelated case, the condition necessary for infinite exchangeability is that $\rho \ge 0$).

Another example of an weakly exchangeable sequence which is not infinitely exchangeable is a multinomial random vector $X_1, X_2, \dotsc, X_k$ with probability vector $(1/k, 1/k, \dotsc, 1/k)$, since the covariances are negative.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.