13
$\begingroup$

I'm interested in obtaining a bootstrapped confidence interval on quantity X, when this quantity is measured 10 times in each of 10 individuals.

One approach is to obtain the mean per individual, then bootstrap the means (eg. resample the means with replacement).

Another approach is to do the following on each iteration of the bootstrapping procedure: within each individual, resample that individual's 10 observations with replacement, then compute a new mean for that individual, and finally compute a new group mean. In this approach, each individual observed in the original data set always contribute to the group mean on each iteration of the bootstrap procedure.

Finally, a third approach is to combine the above two approaches: resample individuals then resample within those individuals. This approach differs from the preceding approach in that it permits the same individual to contribute multiply to the group mean on each iteration, though because each contribution is generated via an independent resampling procedure, these contributions may be expected to vary slightly from eachother.

In practice, I find that these approaches yield different estimates for the confidence interval (ex. with one data set, I find that the third approach yields much larger confidence intervals than the first two approaches), so I'm curious what each might be interpreted to represent.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
7
$\begingroup$

Your first approach is about a between S CI. If you wanted to measure within S then that's the wrong approach.

The second approach would generate a within S CI that would only apply to those 10 individuals.

The last approach is the correct one for the within S CI. Any increases in the CI are because your CI is more representative of a CI that could be applied to the population instead of those 10 S's.

Improve this answer
$\endgroup$
6
$\begingroup$

According to Davison and Hinckley ("Bootstrap methods and their application", 1997, Section 3.8), the third algorithm is conservative. They advocate a fourth approach: simply resampling the subjects.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Interesting, I'll have to look that reference up. Are you sure you mean "fourth" approach? The first approach I list seems to describe "simply resampling the subjects". $\endgroup$
    – Mike Lawrence
    Aug 24, 2010 at 11:17
  • 1
    $\begingroup$ Yes, it does, but it describes resampling the subject means. D&H advocate resampling the subjects and fitting the original model. $\endgroup$
    – Andrew Robinson
    Sep 6, 2010 at 23:00
  • 2
    $\begingroup$ You also might like to see the recently-published: Ren, Shiquan , Lai, Hong , Tong, Wenjing , Aminzadeh, Mostafa , Hou, Xuezhang and Lai, Shenghan(2010) 'Nonparametric bootstrapping for hierarchical data', Journal of Applied Statistics, 37: 9, 1487 — 1498 $\endgroup$
    – Andrew Robinson
    Sep 7, 2010 at 23:28
  • 2
    $\begingroup$ @Mike: resampling the whole clsuter is what survey statisticians do in their bootstraps. That's indeed a different procedure that would only be equivalent to your "first" approach if (i) you are only estimating the mean, and (ii) the data are unweighted and balanced. See also citeulike.org/user/ctacmo/article/1334050, citeulike.org/user/ctacmo/article/1475866, citeulike.org/user/ctacmo/article/582039. $\endgroup$
    – StasK
    Aug 13, 2011 at 15:10
  • $\begingroup$ To update this post with more recent literature see "Saravanan, V., Berman, G. J., & Sober, S. J. (2020). Application of the hierarchical bootstrap to multi-level data in neuroscience. Neurons, behavior, data analysis and theory, 3(5)". They implement cluster bootstrapping in different examples and come to the conclusion that the procedure performs better and more sensitive than procedures that do not account for nested structures. $\endgroup$
    – a.henrietty
    May 22, 2023 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.