# Bootstrapping a t statistic - Rationale and procedure

I was hoping someone could help me out with this. I've seen similar questions on the forum, but I need to know if I've understood the correct rationale and procedure for bootstrapping for my particular need.

In my study, I have two independent groups of equal sample size (22 individuals in each group). I am interested in the difference between these two groups, so I've run a simple t-test. The resulting t-value was 1.39, with a p value of .17. The assumptions of the test look good; Equal variance between groups, and variable is close to normally-distributed (but sample sizes are small).

My supervisor advised me to run "simulations" on the test statistic, since the p value was relatively low (to avoid a type-II error). After reading up on resampling techniques, I've concluded that it is feasible to bootstrap the test-statistic, and then explore the range of t values and associated p values to determine the precision of my estimate. Could someone please verify if this is an appropriate rationale for bootstrapping?

As I understand, one procedure for this would be to: a) randomly extract (With replacement) 22 cases for each of my two Groups, b) perform a t-test on these new samples, c) repeat e.g. 1000 times, d) look/plot the ranges of t and p values, and use these to infer if there is a difference between my groups. I am planning on using R for bootstrapping.

Would this be an appropriate way of bootstrapping in my example? I've read that one can bootstrap data directly, or bootstrap just the test statistic. I guess I really want the latter one, but not sure what the difference between these two approaches are.

• Thanks for the comments! I will try to bootstrap, and depending on where it gets me, consider permutation test or another bootstrapping approach! Jul 6, 2015 at 7:26

Your null hypothesis is that there’s no difference in means between groups, so you should choose a randomized test that checks this. Randomly permute the group assignments (labels) and compare the means (e.g. calculate the $$t$$ statistic). Doing this many times will give you the null distribution of the statistic. Afterward, you could compare how likely is the statistic calculated on real data given the null distribution.
No need to collect the $$p$$-values on the simulated data, just calculate it as
$$p_\text{perm} = \frac{1}{R} \sum_{i=1}^R \,\mathbb{I}\{ t_\text{real} \ge t_{\text{perm}_i} \}$$
where $$R$$ is the number of simulated outcomes. Of course, depending on your hypothesis, choose an appropriate comparison, e.g. $$\ge$$, $$\lt$$, two-tailed difference, etc.