Bootstrap for CIs and Permutation Resampling for Hypothesis Test? In various scattered places online and StackExchange I've read that bootstrap resampling is more appropriate for calculating confidence intervals while permutation resampling is more appropriate for hypothesis testing, but why is this so (or is this generalization incorrect)?
I've seen it argued that this is because bootstrap resampling estimates the true population rather than the distribution under the null, but it seems to me that if confidence interval calculation is valid, the hypothesis testing is also appropriate via seeing if the null hypothesis value lies within the interval. Moreover, we can emulate the null distribution in bootstrapping by shifting our means (subtracting out the sample means and then adding back in the mean under the null).
 A: Simply: The non-parametric bootstrap resampling generates datasets under the alternative hypothesis. Permutation resampling generates datasets under the null hypothesis. The CI probabilities are conditional upon the original sample estimate being the truth. The p-value probability is conditional upon the null hypothesis being the truth. Both procedures can perform the other's probability calculations with some simple to very complex modifications.
A: Permutation tests address nonparametric hypotheses such as equal distributions of two samples or independence of two variables. These tests are exact in theory, and in practice approximation by Monte Carlo simulation is normally unbiased and very precise.
However, by nature they do not address parametric problems. It is true that there are parametric versions of equality and independence problems and permutation tests can be adapted to them. This includes computing confidence intervals, but these are cumbersome and do normally involve more assumptions than the permutation test would otherwise need.
Bootstrap tests on the other hand are usually biased. If a bootstrap and a proper permutation test can be set up for the same problem, there are in most cases good reasons to prefer the permutation test. However there are test problems for which there are no permutation tests (without tedious adaptation that may lose some of the original advantages) but that can be treated using bootstrap.
On the other hand, as permutation tests are by nature nonparametric, they don't lend themselves easily to confidence intervals. 
So in general it's wrong to say that bootstrap is always preferable for CIs and permutation tests are always better for tests, however where a permutation test can be done it is usually better than a bootstrap test, and permutation CIs are tedious and problematic if available at all.      
