Permutation tests seem to provide a promising alternative for the unpaired t-test, requiring fewer assumptions. However, the core assumption of the permutation test, exchangeability, implies homogeneity of variances. So, as for the standard Student's t-test, when the variances and the group sizes are not equal, the TypeI error of the permutation test can be inflated. The following R script demonstrates this
library(perm)
typeIerrors <- 0
reps <- 1000
n1 <- 10
sd1 <- 5
n2 <- 30
sd2 <- 1
for (i in 1:reps){
group1 <- rnorm(n=n1,sd=sd1)
group2 <- rnorm(n=n2,sd=sd2)
permRes <- permTS(group1,group2)$p.value
if (permRes<.05){
typeIerrors <- typeIerrors+1
}
}
cat(sprintf('Type I error rate: %.2f\n',typeIerrors/reps))
Assuming homogeneity of variances seems to be problematic in many cases. The Wikipedia page for the Welch's test (the t-test version, which does not assume homogeneity of variances) notes on this topic: "It is not recommended to pre-test for equal variances and then choose between Student's t-test or Welch's t-test. Rather, Welch's t-test can be applied directly and without any substantial disadvantages to Student's t-test as noted above."
A simple approach to making the permutation test robust to variance heterogeneity might be to use the Welch test statistic as the test statistic. I did some research on this and only found one very short, seemingly unreviewed paper about the topic (http://vetdergikafkas.org/uploads/pdf/pdf_KVFD_779.pdf). Do you know other articles discussing the permutation version of the Welch's t-test? Optimally, discussing why or why not it is a good approach.
I found other, seemingly more complex approaches for the problem of making permutation tests robust towards unequal variances (for example, 1, and 2).
springerlink.com
. But, it seems that your two examples (in the last sentence) both point to the same paper. Perhaps you could take a look, whenever possible… $\endgroup$