# Using non-parametric bootstrap to assess relative difference between means of two samples

Let's say we have 2 samples, A and B. I'd like to find out which distribution has a higher mean by using non-parametric bootstrap.

My procedure goes like this:

1. Calculate the mean of a resample 1 from sample A: $$\bar{x}*$$
2. Calculate the mean of a resample 1 from sample B: $$\bar{y}*$$
3. Calculate relative difference between $$(\bar{x*}-\bar{y}*) / \bar{x*}$$.
4. Repeat steps 1-3 1000 times to get a distribution of relative differences.

Can interpret the results like that: If the 95% CI doesn't include 0, then the relative difference is significant (alpha=0.05)?

To me, the procedure looks like we are interpreting non-parametric bootstrap as Bayesian model that has nothing to do with classical frequentist statistical significance since in that case we should bootstrap scenario in which null hypothesis holds (like here or here or permutation test) and then compare it to the actual relative difference.

• I believe that one could do a permutation test for the difference of two means. I don't know why we would bootstrap re-sample in this situation.
– JTH
Commented Nov 10, 2021 at 21:36
• @JTH I agree a permutation test would be a standard approach. However, I'm interested in interpretation of the approach described above. Commented Nov 11, 2021 at 3:05
• Why would that be a bayesian interpretation? Significance levels and confidence intervals are closely related, and one can be obtained from the other for any test. This is as frequentist as it can get. Commented Nov 11, 2021 at 11:13