I'm wondering about why we do a particular thing (and not another) when conducting bootstrapped hypothesis tests.
My understanding of bootstrapped hypothesis tests is this (based on this helpful explanation):
- State a null and alternative hypothesis. H0: mean = 50 and Ha: mean <> 50, for example
- Collect a sample of data. Let's say the mean of this sample is 62.
- From each of the observations in our sample, subtract the mean of our sample. So in this example, subtract 62 from each observation. Then add the mean value under the null hypothesis to each of our observations. So in this example, add 50 to each of our observations. Now we've got a sample that's centered on our null hypothesis mean.
- Resample this shifted sample with replacement a bunch of times. Each time, recalculate the mean, so we have a bunch of bootstrapped means. This is our bootstrapped distribution under the null hypothesis.
- Compare our original observed sample mean (62) to our bootstrapped null distribution. See how many bootstrapped means are at least as extreme as 62 is. This gives us a p-value.
Ok here's my question:
Is it ok to do what I'm about to describe below? If not, why not? And if not, is there a version of this that's ok to do?
- State a null and alternative hypothesis. H0: mean = 50 and Ha: mean <> 50, for example. Same as before.
- Collect a sample of data. Let's say the mean of this sample is 62. Same as before
- Resample our original sample with replacement a bunch of times. Each time, recalculate the mean, so we have a bunch of bootstrapped means. This creates a bootstrapped distribution (but not one centered on the null hypothesis value). This is different from above
- Compare our null hypothesis mean (50) to our bootstrapped distribution (centered roughly on 62). See how many bootstrapped means are at least as extreme as 50 is. This gives us a p-value. This is different from above
Again: Is it ok to do that? If not, why not? And if not, is there a version of this that's ok to do?
Any help would be much appreciated. Happy to clarify anything. Thanks so much!
PS: I've read this related post, but my question is why it's not acceptable to use the empirical distribution as the reference distribution when calculating the pvalue.