# Choosing N when doing t-tests on bootstrapped samples

Specifically the Welch's T-Test, but probably any T-Test, requires a value for Mean, Variance and N to be used to calculate the T-statistic and the degrees of freedom. I am concerned about these values if the sample has been bootstrapped (re-sampled with replacement) before using the T-Test. Specifically, I'm concerned that the arbitrary choice of how many times to bootstrap (10K, 20K, 100K) will change the T-statistic even though the underlying distribution is the same.

For instance, if I have 100 sample values with an unknown distribution and I re-sample with replacement 10,000 times, then I use those 10,000 samples to calculate a mean, variance, and N for a t-test.

N: Should I use the sample size before the bootstrap or after the bootstrap? I'm concerned that my choice of how many times to run the bootstrap (10,000 is an arbitrary number) will make the distribution look more accurate than it is. Should I use n=10,000 or N=100?

Full disclosure: I have a fairly good knowledge of statistics, enough to use the terms, but perhaps not enough to ensure I have used them 100% correctly. Please excuse confusions of terminology.

• In what sense is variance "cumulative" in a way that any other statistic, such as the mean, is not?
– whuber
Jan 12 '12 at 16:39
• My mistake. When looking at the equation for variance I didn't notice that it adjusted for the sample size. I will amend the question. Jan 12 '12 at 17:08
• Just to clarify, are you calculating the t-statistic once for each bootstrapped sample? Or are you calculating some other statistic $\theta_i$ once for each of the $N$ bootstrapped samples and a single t-statistic on the $N$ $\theta_i$? Jan 13 '12 at 0:01
• @jbowman Not sure as to your question, but in an attempt to clarify, I'm using a bootstrapped sample and using a T-Test to compare means with some other sample. If I had two samples (neither of which are bootstrapped), then the sample sizes to use is straightforward in a T-Test. I'm asking about which sample size to use when one of the samples has been bootstrapped. Jan 13 '12 at 18:26