# Bootstrap confidence interval in within-subjects study

I have a within-subjects study where for each participant I have 10 datapoints from the control condition and 10 datapoints from the treatment condition. The treatment effect is calculated as [mean of outcomes from treatment condition] - [mean of outcomes from control condition].

I am trying to use bootstrap to calculate a confidence interval around the treatment effect. I've only seen bootstrap used on paired data when each participant only has 1 datapoint from the treatment and 1 datapoint from the control. In that setting, you can simply bootstrap the distribution of paired differences. But in my setting, this does not work because there are 10 datapoints from each participant (and individual datapoints do not have natural pairs).

Is the following a valid procedure?

Repeat n times:
For each participant:
Bootstrap sample 10 datapoints (from their 20) to be their control datapoints
Bootstrap sample 10 datapoints (from their 20) to be their treatment datapoints
Calculate the test statistic
Calculate the p-value using the distribution of n test statstics


In brief, the answer is yes...this protocol would work.

Here are a couple of variations you might consider.

1. For each participant, sample the original means (control and treatment from the actual 10 values) and randomly assign these as treatment and control; calculate the test statistic; repeat n times and find the p-value. This will ignore the variability within the different 10 trials, but will be more like the "conventional" bootstrap for a matched-pairs design.

2. Use the strategy you indicated here, but sample without replacement...so 10 of the measurements are assigned control and the remaining 10 are assigned treatment; calculate the test statistic; repeat n times and find the p-value. This will incorporate the within variability, but it forces (some of) the marginal means to match the original data set.

3. Use the strategy you indicated here...and you can either sample the 20 with or without replacement at each stage. It can be argued that either of these strategies would best capture the null hypothesis that the distributions are the same at control and treatment.

Hope this helps.

• Thank you, that helps!
– tree
Commented May 19, 2023 at 21:35