3
$\begingroup$

Specifically the Welch's T-Test, but probably any T-Test, requires a value for Mean, Variance and N to be used to calculate the T-statistic and the degrees of freedom. I am concerned about these values if the sample has been bootstrapped (re-sampled with replacement) before using the T-Test. Specifically, I'm concerned that the arbitrary choice of how many times to bootstrap (10K, 20K, 100K) will change the T-statistic even though the underlying distribution is the same.

For instance, if I have 100 sample values with an unknown distribution and I re-sample with replacement 10,000 times, then I use those 10,000 samples to calculate a mean, variance, and N for a t-test.

N: Should I use the sample size before the bootstrap or after the bootstrap? I'm concerned that my choice of how many times to run the bootstrap (10,000 is an arbitrary number) will make the distribution look more accurate than it is. Should I use n=10,000 or N=100?

Full disclosure: I have a fairly good knowledge of statistics, enough to use the terms, but perhaps not enough to ensure I have used them 100% correctly. Please excuse confusions of terminology.

Improve this question
$\endgroup$
4
  • $\begingroup$ In what sense is variance "cumulative" in a way that any other statistic, such as the mean, is not? $\endgroup$
    – whuber ♦
    Commented Jan 12, 2012 at 16:39
  • $\begingroup$ My mistake. When looking at the equation for variance I didn't notice that it adjusted for the sample size. I will amend the question. $\endgroup$
    – user548084
    Commented Jan 12, 2012 at 17:08
  • $\begingroup$ Just to clarify, are you calculating the t-statistic once for each bootstrapped sample? Or are you calculating some other statistic $\theta_i$ once for each of the $N$ bootstrapped samples and a single t-statistic on the $N$ $\theta_i$? $\endgroup$
    – jbowman
    Commented Jan 13, 2012 at 0:01
  • $\begingroup$ @jbowman Not sure as to your question, but in an attempt to clarify, I'm using a bootstrapped sample and using a T-Test to compare means with some other sample. If I had two samples (neither of which are bootstrapped), then the sample sizes to use is straightforward in a T-Test. I'm asking about which sample size to use when one of the samples has been bootstrapped. $\endgroup$
    – user548084
    Commented Jan 13, 2012 at 18:26

1 Answer 1

Reset to default
5
$\begingroup$

Normally each of your bootstrap samples would be the same size as your original. So if you have 100 in your original sample, your bootstrap samples should also be size 100 (or, some argue, 99). Not 10000. So then each of your samples would calculate the T using the sample size of 100. You can do 10000 bootstraps if you like (probably excessive for most questions) but I don't see much value in having bootstrap samples that are each bigger than the original sample. The aim of the bootstrap is to resemble the original sampling process as much as possible, hence you use the same size.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.