9
$\begingroup$

I have been working on the uncertainty associated with a quantity calculated from a Monte Carlo project. Normally I would use the bootstrap method by resampling with replacement, for a couple of technical reasons that is not particularly easy here. It was suggested that I just break up my MC data set and perform the experiment with these subsets and find the uncertainty that way. I have in the past come across references to bootstrapping with only a subset of the original dataset.

Can someone point me to a tutorial on this or explain briefly how it is different to bootstrapping with replacement and just setting the number of samples to a fraction of the total size. I would be particularly interested in a method that meant that $n$ could be different for each subsample, this would make my analysis much more simple.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ I think we need to hear more about why you can't just resample with replacement for the main statistic of interest. Why is it possible to do this with subsets but not the original dataset? Is it just a matter of scale? $\endgroup$
    – Peter Ellis
    Commented Aug 8, 2012 at 20:15
  • $\begingroup$ The MC stores a lot of other data in a storage tree (~ few GB per MC run) and the trees are not designed to accessed randomly so there is an issue of scale although if this was the only issue I could probably dump it in the RAM of a reasonably beefy desktop. The statistic also depends on a number of parameters of which my statistic is a function so to do full resampling with replacement would require me to break up my data structure. If the simple answer to my question is no then I'll knuckle down and get coding (and find a bigger desktop) although a nice elegant solution would be of interest. $\endgroup$
    – Bowler
    Commented Aug 8, 2012 at 20:23
  • $\begingroup$ +1 to Peter -- the problem statement is not even approaching the level of clarity at which somebody could point to a reasonable solution. If you want a good answer, come up with a good question. You might have reasons to protect confidentiality, or commerical interests, or an extremely hot research topic that you don't want to be scooped of, but without a clear explanation what your problem is, the answers will likely be irrelevant to you. $\endgroup$
    – StasK
    Commented Aug 9, 2012 at 11:02
  • 1
    $\begingroup$ If anything I think the context probably diluted my question which was simply about alternatives to the bootstrap that don't require the number of samples per iteration to be equal to the number of data. I think details of nuclear MC codes would not be enlightening. $\endgroup$
    – Bowler
    Commented Aug 9, 2012 at 14:21
  • $\begingroup$ @StasK I have to agree with Bowler. I think the question was clear enough to answer. The details would only help satisfy our curiosity as to why subsampling is necessary. $\endgroup$
    – Michael R. Chernick
    Commented Aug 9, 2012 at 14:37

1 Answer 1

Reset to default
17
$\begingroup$

There are two methods related to your question. One is the m out of n bootstrap and the other is random subsampling. In his original proposal Efron picked the bootstrap sample size to be the same as the original sample size. There was no specific requirement to do that but the idea was to mimic random sampling from the population as closely as possible. However there are situations where this ordinary bootstrap is inconsistent. Bickel and Ren among others showed that taking a smaller sample size m can lead to consistent results. This works asymptotically with m and n both tending to infinity but at a rate so that m/n goes to 0. Random subsampling was introduced by Hartigan and McCarthy in the late 1960s about a decade before the bootstrap. It uses a procedure of randomly sampling subsets of the original sample. It may be that you could take either of these approaches with your data.

For information on the m out of n bootstrap you can consult either of the following books that I authored/co-authored:

An Introduction to Bootstrap Methods with Applications to R

Bootstrap Methods: A Guide for Practitioners and Researchers

This book by Politis, Romano and Wolf goes into random subsampling in great detail:

Subsampling

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ thanks with a bit of extra coding I think m out of n is what I'm looking for, never realised there was so much to the bootstrap. $\endgroup$
    – Bowler
    Commented Aug 9, 2012 at 14:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.