What is an example of data where the permutation test succeeds but a normal t-test fails? In literature, I normally see authors use a two sample permutation test on normal data to show that it works as well as the two sample t-test. However, the real power for permutation tests should be the non-parametric properties. 
Does anyone have a good example of non-normal two sample data that fails the t-test but where the permutation test succeeds?
 A: In general, situations in which there is not enough data for the difference between the two sample means to have converged to something "near" the t distribution will cause the t-test to fail, in the sense of not having close to the specified probability of rejecting a true null hypothesis.
Let's assume we are drawing two samples from some distribution that does not have a mean or variance, e.g, a Cauchy distribution.   In that case, the two-sample t-test will fail for obvious reasons, but the permutation test will be unaffected.  Here's an example showing, via simulation, roughly the distributions of the p-values observed from the t-test and the permutation test when comparing the sample means of two standard Cauchy variates with sample sizes of 10:
    p_value_t <- rep(0, 1000)
    p_value_perm <- rep(0, 1000)
    
    delta <- rep(0, 1000)
    
    for (i in seq_along(p_value_t)) {
       x1 <- rcauchy(10)
       x2 <- rcauchy(10)
       
       t_value <- (mean(x1) - mean(x2)) / sqrt(var(x1)/10 + 
                    var(x2)/10)
       p_value_t[i] <- pt(t_value, 18)
       
       x_all <- c(x1, x2)
       for (j in seq_along(delta)) {
          x_all <- sample(x_all)
          delta[j] <- mean(x_all[1:10]) - mean(x_all[11:20])
       }
       p_value_perm[i] <- mean((mean(x1) - mean(x2)) < delta)
    }
    
    hist(p_value_t, main = "Histogram of t-test based p-values", 
      xlab="p-value", ylab="Observed frequency")
    hist(p_value_perm, main = 
      "Histogram of permutation based p-values", xlab="p-value", 
       ylab="Observed frequency")

... and the resulting histograms:


Clearly the p-values from the t-test do not have the desired Uniform distribution, but those from the permutation test look pretty good in this respect.
We have made this example rather extreme, as the Cauchy does not have a mean or variance, but the fundamental principle holds: the t-test works reasonably well when the sample sizes are large enough so that the distribution of the difference between the sample means actually has, roughly, a t distribution with the calculated degrees of freedom, but breaks down when this assumption is violated more severely in practice.
A: I think a good example is the space shuttle data as given in the textbook $\it{The \ Statistical \ Sleuth}.$ The number of O-ring incidents is given by launch temperature as 
Temp $ \ \ \ \ \ \ \ \ \ \ \ \ $ Number of O-Ring Incidents
$\lt 65^{\circ} \ \ \ \ \ \ \ \ \ \ \ \ \ \ 1,1,1,3$
$\gt 65^{\circ} \ \ \ \ \ \ \ \ \ \ \ \ \ \ 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,2$ 
A Welch's t-test on this data gives a p-value of $0.0853$
The permutation test gives $0.0099$ 
